Storm-2139 Azure OpenAI Guardrail Bypass

Summary

Storm-2139 built custom jailbreak tooling to bypass guardrails on Azure OpenAI Services, allowing users to generate harmful synthetic content. Microsoft reported that members of Storm-2139 scraped exposed customer credentials from public sources and used them to access accounts for generative AI services. The group developed and operated tools and services that bypassed safety safeguards, modified service capabilities, and enabled end users to generate harmful and illicit content, including non-consensual intimate images of celebrities and other sexually explicit content. The operation included creators who developed illicit tools, providers who modified and supplied those tools, and users who generated prohibited content. Microsoft pursued civil legal action. Target: Microsoft Azure OpenAI Service Actor: Storm-2139 Reporter: Microsoft Procedure: 1. Storm-2139 creators developed a tool called de3u to facilitate unauthorized use of generative AI services and bypass safeguards. 2. Storm-2139 acquired infrastructure to support a service that sold access to the jailbroken Azure OpenAI Service. 3. Storm-2139 staged and operated a reverse proxy service to allow other malicious users to interact with abused generative AI services. 4. Storm-2139 used exposed customer credentials scraped from public sources to access valid accounts for generative AI services. 5. The stolen credentials provided access Azure OpenAI Service, allowing the actors and their customers to submit prompts and generate content. Storm-2139's de3u tool was used as the frontend for this access. 6. Storm-2139 deliberately bypassed Azure OpenAI Service safeguards and content filters to generate prohibited outputs. Microsoft reported that the actors iterated on blocked prompts, substituted celebrity descriptions, and used altered wording or technical notation to evade filters. 7. End users generated abusive synthetic imagery, including non-consensual intimate images of celebrities and other sexually explicit, misogynistic, violent, or hateful content. 8. The generated abusive imagery could cause direct harm to depicted individuals.

Deployers

• Microsoft Azure OpenAI Service

Tags

• MITRE ATLAS
• AML.CS0057