Cursor, makers of an AI-powered code editor, recently found their business undermined by an AI-powered chatbot that invented a user-hostile policy.
When the user contacted Cursor support, an agent named “Sam” told them it was expected behavior under a new policy. But no such policy existed, and Sam was a bot. The AI model made the policy up, sparking a wave of complaints and cancellation threats documented on Hacker News and Reddit.
Last year, Air Canada learned that they were responsible for their chatbot. Now, Cursor is learning the same lesson.
Previously in confident wrongness: Warning: AI Does Not Understand Satire
