AI Model Tampering via Supply Chain Attack

Summary

Researchers at Trend Micro, Inc. used service indexing portals and web searching tools to identify over 8,000 misconfigured private container registries exposed on the internet. Approximately 70% of the registries also had overly permissive access controls that allowed write access. In their analysis, the researchers found over 1,000 unique AI models embedded in private container images within these open registries that could be pulled without authentication. This exposure could allow adversaries to download, inspect, and modify container contents, including sensitive AI model files. This is an exposure of valuable intellectual property which could be stolen by an adversary. Compromised images could also be pushed to the registry, leading to a supply chain attack, allowing malicious actors to compromise the integrity of AI models used in production systems. Target: Private Container Registries Actor: Trend Micro Nebula Cloud Research Team Procedure: 1. The Trend Micro researchers used service indexing portals and web searching tools to identify over 8,000 private container registries exposed on the internet. Approximately 70% of the registries had overly permissive access controls, allowing write permissions. The private container registries encompassed both independently hosted registries and registries deployed on Cloud Service Providers (CSPs). The registries were exposed due to some combination of: - Misconfiguration leading to public access of private registry, - Lack of proper authentication and authorization mechanisms, and/or - Insufficient network segmentation and access controls 2. The researchers were able to exploit the misconfigured registries to pull container images without requiring authentication. In total, researchers pulled several terabytes of data containing over 20,000 images. 3. The researchers found 1,453 unique AI models embedded in the private container images. Around half were in the Open Neural Network Exchange (ONNX) format. 4. This gave the researchers full access to the models. Models for a variety of use cases were identified, including: - ID Recognition - Face Recognition - Object Recognition - Various Natural Language Processing Tasks 5. With full access to the model(s), an adversary has an organization's valuable intellectual property. 6. With full access to the model weights, an adversary could manipulate the weights to cause misclassifications or otherwise degrade performance. 7. With full access to the model, an adversary could modify the architecture to change the behavior. 8. Because many of the misconfigured container registries allowed write access, the adversary's container image with the manipulated model could be pushed with the same name and tag as the original. This compromises the victim's AI supply chain, where automated CI/CD pipelines could pull the adversary's images.

Deployers

• Private Container Registries

Tags

• MITRE ATLAS
• AML.CS0028