Skip to content
BiasMedium

High-Toxicity Assessed on Text Involving Women and Minority Groups

Occurred
Feb 27, 2017
Published
Feb 10, 2021
Source
AI Incident Database

Summary

Google's Perspective API, which assigns a toxicity score to online text, seems to award higher toxicity scores to content involving non-white, male, Christian, heterosexual phrases.

Developers

  • Google

Deployers

  • Google

Harmed parties

  • Women
  • Minority groups
  • Women and girls

Classification

Category confidence: 95%
Severity confidence: 90%

Rationale: The API exhibits discriminatory scoring against minority groups, representing a clear bias incident without major legal or physical harm.

Classifier: Qwen/Qwen3.5-35B-A3B-FP8

Visit source