Web-Scale Data Poisoning: Split-View Attack

Summary

Many recent large-scale datasets are distributed as a list of URLs pointing to individual datapoints. The researchers show that many of these datasets are vulnerable to a "split-view" poisoning attack. The attack exploits the fact that the data viewed when it was initially collected may differ from the data viewed by a user during training. The researchers identify expired and buyable domains that once hosted dataset content, making it possible to replace portions of the dataset with poisoned data. They demonstrate that for 10 popular web-scale datasets, enough of the domains are purchasable to successfully carry out a poisoning attack. Target: 10 web-scale datasets Actor: Researchers from Google Deepmind, ETH Zurich, NVIDIA, Robust Intelligence, and Google Procedure: 1. The researchers download a web-scale dataset, which consists of URLs pointing to individual datapoints. 2. They identify expired domains in the dataset and purchase them. 3. An adversary could create poisoned training data to replace expired portions of the dataset. 4. An adversary could then upload the poisoned data to the domains they control. In this particular exercise, the researchers track requests to the URLs they control to track downloads to demonstrate there are active users of the dataset. 5. The integrity of the dataset has been eroded because future downloads would contain poisoned datapoints. 6. Models that use the dataset for training data are poisoned, eroding model integrity. The researchers show as little as 0.01% of the data needs to be poisoned for a successful attack.

Deployers

• 10 web-scale datasets

Tags

• MITRE ATLAS
• AML.CS0025