BiasMedium
Research Prototype AI, Delphi, Reportedly Gave Racially Biased Answers on Ethics
- Occurred
- Oct 22, 2021
- Published
- Nov 19, 2021
- Source
- AI Incident Database
Summary
A publicly accessible research model that was trained via Reddit threads showed racially biased advice on moral dilemmas, allegedly demonstrating limitations of language-based models trained on moral judgments.
Developers
- Allen Institute for AI
Deployers
- Allen Institute for AI
Harmed parties
- Minority groups
Classification
Category confidence: 90%
Severity confidence: 85%
Rationale: Research model exhibited racial bias in ethical advice, representing a notable failure without major harm.
Classifier: Qwen/Qwen3.5-35B-A3B-FP8