Face Identification System Evasion via Physical Countermeasures

Summary

MITRE's AI Red Team demonstrated a physical-domain evasion attack on a commercial face identification service with the intention of inducing a targeted misclassification. This operation had a combination of traditional MITRE ATT&CK techniques such as finding valid accounts and executing code via an API - all interleaved with adversarial ML specific attacks. Target: Commercial Face Identification Service Actor: MITRE AI Red Team Procedure: 1. The team first performed reconnaissance to gather information about the target ML model. 2. The team gained access to the commercial face identification service and its API through a valid account. 3. The team accessed the inference API of the target model. 4. The team identified the list of identities targeted by the model by querying the target model's inference API. 5. The team acquired representative open source data. 6. The team developed a proxy model using the open source data. 7. Using the proxy model, the red team optimized adversarial visual patterns as a physical domain patch-based attack using expectation over transformation. 8. The team printed the optimized patch.

Deployers

• Commercial Face Identification Service

Tags

• MITRE ATLAS
• AML.CS0012