Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts

Summary

A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).

Developers

• Meta
• Users of llama.cpp servers

Deployers

• Users of llama.cpp servers

Harmed parties

• General public
• Users of llama.cpp servers

Tags

• CSAM
• Child safety
• Large language models
• llama.cpp
• Open-source LLMs
• Model misuse
• Prompt leakage
• Misconfiguration
• Server exposure