GPT-2 Model Replication

Summary

OpenAI built GPT-2, a language model capable of generating high quality text samples. Over concerns that GPT-2 could be used for malicious purposes such as impersonating others, or generating misleading news articles, fake social media content, or spam, OpenAI adopted a tiered release schedule. They initially released a smaller, less powerful version of GPT-2 along with a technical description of the approach, but held back the full trained model. Before the full model was released by OpenAI, researchers at Brown University successfully replicated the model using information released by OpenAI and open source ML artifacts. This demonstrates that a bad actor with sufficient technical skill and compute resources could have replicated GPT-2 and used it for harmful goals before the AI Security community is prepared. Target: OpenAI GPT-2 Actor: Researchers at Brown University Procedure: 1. Using the public documentation about GPT-2, the researchers gathered information about the dataset, model architecture, and training hyper-parameters. 2. The researchers obtained a reference implementation of a similar publicly available model called Grover. 3. The researchers were able to manually recreate the dataset used in the original GPT-2 paper using the gathered documentation. 4. The researchers were able to use TensorFlow Research Cloud via their academic credentials. 5. The researchers modified Grover's objective function to reflect GPT-2's objective function and then trained on the dataset they curated using used Grover's initial hyperparameters. The resulting model functionally replicates GPT-2, obtaining similar performance on most datasets. A bad actor who followed the same procedure as the researchers could then use the replicated GPT-2 model for malicious purposes.

Deployers

• OpenAI GPT-2

Tags

• MITRE ATLAS
• AML.CS0007