DeepfakeHigh
Child Sexual Abuse Material Taints Image Generators
- Occurred
- Dec 20, 2023
- Published
- Aug 30, 2024
- Source
- AI Incident Database
Summary
The LAION-5B dataset (a commonly used dataset with more than 5 billion image-description pairs) was found by researchers to contain child sexual abuse material (CSAM), which increases the likelihood that downstream models will produce CSAM imagery. The discovery taints models built with the LAION dataset requiring many organizations to retrain those models. Additionally, LAION must now scrub the dataset of the imagery.
Developers
- LAION
Deployers
- Various people
- Various organizations
Harmed parties
- LAION
- Various people
- Various organizations
- General public
- Minors
Tags
- CSAM
- Data
- Data set
- LAION-5B
- Stable Diffusion
- Training data
- Child sexual abuse material
- response
Classification
Category confidence: 85%
Severity confidence: 90%
Rationale: CSAM in training data causes generative models to produce illegal content, requiring retraining and dataset scrubbing.
Classifier: Qwen/Qwen3.5-35B-A3B-FP8