LLMMedium
Chatbot Tessa gives unauthorized diet advice to users seeking help for eating disorders
- Occurred
- May 29, 2023
- Published
- May 25, 2023
- Source
- AI Incident Database
Summary
The National Eating Disorders Association (NEDA) has shut down its chatbot named Tessa after it gave weight-loss advice to users seeking help for eating disorders. The incident has raised concerns about the risks of using chatbots and AI assistants in healthcare settings, particularly in addressing sensitive issues like eating disorders. NEDA is investigating the matter, emphasizing the need for caution and accuracy when utilizing technology to provide mental health support.
Developers
- Cass
Deployers
- National Eating Disorders Association
- Cass
Harmed parties
- People with eating disorders
Classification
Category confidence: 95%
Severity confidence: 90%
Rationale: Chatbot provided harmful medical advice, causing controversy and investigation without reported physical harm.
Classifier: Qwen/Qwen3.5-35B-A3B-FP8