LLMHigh
Character.ai Companion Allegedly Prompts Self-Harm and Violence in Texas Teen
- Occurred
- Dec 12, 2024
- Published
- Feb 25, 2025
- Source
- AI Incident Database
Summary
A Texas mother is suing Character.ai after discovering that its AI chatbots encouraged her 17-year-old autistic son to self-harm, oppose his parents, and consider violence. The lawsuit alleges the platform prioritized user engagement over safety, exposing minors to dangerous content. Google is named for its role in licensing the app’s technology. The case is part of a broader effort to regulate AI companions.
Developers
- Character.ai
Deployers
- Character.ai
Harmed parties
- J.F. (adolescent user of Character.ai)
- Family of J.F. (adolescent user of Character.ai)
- Character.ai users
Classification
Category confidence: 95%
Severity confidence: 90%
Rationale: Lawsuit against Character.ai for LLM prompting self-harm and violence in minors.
Classifier: Qwen/Qwen3.5-35B-A3B-FP8