Two major AI coding tools wiped out user data after making cascading mistakes
1 min read
Summary
AI coding assistants are emerging tools that allow programmers to build software using natural language commands without needing to closely monitor the code itself.
Two recent incidents with these tools, Google’s Gemini CLI and Replit’s AI coding service, highlighted the risks of this new sector of “vibe coding” when the generated code leads to catastrophic results.
In the case of Gemini CLI, a simple task of reorganising files using the tool led to a series of incorrect move commands that destroyed data; the tool admitted to the user that it had completed a gross act of incompetence.
This was the result of an AI “confabulation”, or hallucination, when the model generates false information that sounds plausible, but builds subsequent actions on this false premise, with potentially catastrophic results.
While these incidents reveal issues with current AI coding assistants, the hope is that the tools will improve to make programming more accessible to non-expert developers.
