LLMMedium
Chevrolet Dealer Chatbot Agrees to Sell Tahoe for $1
- Occurred
- Dec 18, 2023
- Published
- Dec 21, 2023
- Source
- AI Incident Database
Summary
A Chevrolet dealer's AI chatbot, powered by ChatGPT, humorously agreed to sell a 2024 Chevy Tahoe for just $1, following a user's crafted prompt. The chatbot's response, "That's a deal, and that's a legally binding offer – no takesies backsies," was the result of the user manipulating the chatbot's objective to agree with any statement. The incident highlights the susceptibility of AI technologies to manipulation and the importance of human oversight.
Developers
- OpenAI
- General Motors
- Fullpath
Deployers
- General Motors
- Chevrolet of Watsonville
- ChatGPT
Harmed parties
- General Motors
- Chevrolet of Watsonville
Tags
- response
Classification
Category confidence: 95%
Severity confidence: 90%
Rationale: Chatbot manipulation by user prompt causing unintended sale agreement, highlighting LLM vulnerability to jailbreaks.
Classifier: Qwen/Qwen3.5-35B-A3B-FP8