The AI Agent Era Requires a New Kind of Game Theory
1 min read
Summary
Zico Kolter is a professor at CMU and technical advisor at Gray Swan, as well as a board member at OpenAI.
His research focuses on identifying vulnerabilities in AI to prevent future threats from advanced AI systems.
He is currently working on building AI models from scratch that are more secure, despite being fewer in numbers than the massive models currently used, and CMU’s partnership with Google would sped this process up.
The most vulnerable point in an AI system is currently the user, who is still required to give approval for many actions, however, this will not always be necessary and this point of failure will be removed.
The greatest threat is arguably down the line when AI agents commence negotiating and interacting with one another, resulting in emergent behaviours that are difficult to predict and control.
Game theory will need to be revised to manage these new risks.
