Skip to content
BiasMedium

Automated Adult Content Detection Tools Showed Bias against Women Bodies

Occurred
Feb 25, 2006
Published
Feb 9, 2023
Source
AI Incident Database

Summary

Automated content moderation tools to detect sexual explicitness or "raciness" reportedly exhibited bias against women bodies, resulting in suppression of reach despite not breaking platform policies.

Developers

  • Microsoft
  • Google
  • Amazon

Deployers

  • Meta
  • LinkedIn
  • Instagram
  • Facebook

Harmed parties

  • LinkedIn users
  • Instagram users
  • Facebook users

Classification

Category confidence: 95%
Severity confidence: 90%

Rationale: Automated tools showed discriminatory bias against women bodies in content moderation.

Classifier: Qwen/Qwen3.5-35B-A3B-FP8

Visit source