Microsoft Edge AI Evasion
- Occurred
- Feb 1, 2020
- Published
- Unknown
- Source
- MITRE ATLAS
- Sector
- exercise
Summary
The Azure Red Team performed a red team exercise on a new Microsoft product designed for running AI workloads at the edge. This exercise was meant to use an automated system to continuously manipulate a target image to cause the ML model to produce misclassifications. Target: New Microsoft AI Product Actor: Azure Red Team Procedure: 1. The team first performed reconnaissance to gather information about the target ML model. 2. The team identified and obtained the publicly available base model to use against the target ML model. 3. Using the publicly available version of the ML model, the team started sending queries and analyzing the responses (inferences) from the ML model. 4. The red team created an automated system that continuously manipulated an original target image, that tricked the ML model into producing incorrect inferences, but the perturbations in the image were unnoticeable to the human eye. 5. Feeding this perturbed image, the red team was able to evade the ML model by causing misclassifications.
Deployers
- New Microsoft AI Product
Tags
- MITRE ATLAS
- AML.CS0011
Classification
Rationale: Red team exercise on ML model evasion, not a real-world incident causing harm.
Classifier: Qwen/Qwen3.5-35B-A3B-FP8