Organization Confusion on Hugging Face

Summary

[threlfall_hax](https://5stars217.github.io/), a security researcher, created organization accounts on Hugging Face, a public model repository, that impersonated real organizations. These false Hugging Face organization accounts looked legitimate so individuals from the impersonated organizations requested to join, believing the accounts to be an official site for employees to share models. This gave the researcher full access to any AI models uploaded by the employees, including the ability to replace models with malicious versions. The researcher demonstrated that they could embed malware into an AI model that provided them access to the victim organization's environment. From there, threat actors could execute a range of damaging attacks such as intellectual property theft or poisoning other AI models within the victim's environment. Target: Hugging Face users Actor: threlfall_hax Procedure: 1. The researcher registered an unverified "organization" account on Hugging Face that squats on the namespace of a targeted company. 2. Employees of the targeted company found and joined the fake Hugging Face organization. Since the organization account name is matches or appears to match the real organization, the employees were fooled into believing the account was official. 3. The employees made use of the Hugging Face organizaion and uploaded private models. As owner of the Hugging Face account, the researcher has full read and write access to all of these uploaded models. 4. With full access to the model, an adversary could steal valuable intellectual property in the form of AI models. 5. The researcher embedded [Sliver](https://github.com/BishopFox/sliver), an open source C2 server, into the target model. They added a `Lambda` layer to the model, which allows for arbitrary code to be run, and used an `exec()` call to execute the Sliver payload. 6. The researcher re-uploaded the manipulated model to the Hugging Face repository. 7. The victim's AI model supply chain is now compromised. Users of the model repository will receive the adversary's model with embedded malware. 8. When any future user loads the model, the model automatically executes the adversary's payload.

Deployers

• Hugging Face users

Tags

• MITRE ATLAS
• AML.CS0027