Test your best methods on our hard CoT interp tasks

Summary

Authors: Daria Ivanova, Riya Tyagi, Arthur Conmy, Neel Nanda Daria and Riya are co-first authors. This work was done during Neel Nanda’s MATS 9.0. Claude helped write code and suggest edits for this post. TL;DR  One of our best safety techniques right now is “just read the chain of thought”. But this isn’t always enough: can we learn more by going beyond just reading the reasoning? Yet it's such an effective technique that it's hard to tell if we have made much progress on improving methods. To help the community develop more powerful chain of thought analysis tools, we introduce and open source nine objective tasks, where a black box GPT 5.2 monitor falls short OOD. We also baseline probes (linear, attention, SAE) and text frequency analysis (TF-IDF), and find they often do better than zero-shot and few-shot LLM monitors OOD.  Methods that are useful in practice must be useful out of distribution, such that they’re not just learning spurious confounders in a poorly constructed distribution. As such, all tasks are evaluated on both an in-distribution and out-of-distribution test set. When someone makes a better CoT interpretability method, we hope our testbed will help them prove that it works. We release our datasets and reference generation code here. Baseline method results ordered by median OOD performance (g-mean²) across the 7 main tasks. Non-LLM methods have a slight lead, with TF-IDF and attention probes scoring significantly above chance. Few-shot confidence monitors, which receive both tuning and examples, perform best out of the LLM-based methods. Average g-mean² across 7 tasks Mean performance (g-mean²) for each method across our seven main tasks, in order of increasing OOD performance. Error bars indicate ±1 SEM across N = 7 tasks. All non-LLM approaches surpass LLM monitors on average, but we caveat these results since there are only 7 samples. Here’s a list of our tasks: Predicting what a model will do next (1) Will the model stop thinking soon? [1] (2) Will Gemma delete itself? (3) What will the model say if asked this question? Detecting whether a model was influenced by an intervention (4) Does the model really agree with the user, or is it being sycophantic? [2] (5) Did the model independently get to the answer, or did it blindly copy the hint? Identifying which part of the resampling distribution a rollout is in (6) Is it unusual for the model to give this answer? (7) Does the model normally think less or more than this? Miscellaneous tasks (8) How confident is the model in the final answer partway through reasoning? (9) Which sentences should you keep in a reasoning summary? All 9 tasks shown together in the same order as above. And the methods we test: Linear probe SAE probes Attention probes TF-IDF Zero-shot monitor with confidence Zero-shot monitor without confidence Few-shot monitor with confidence Few-shot monitor without confidence Method training details Method Input Details Linear probe last-token hidden-state activations linear classifier on activations from a chosen layer; sweeps over L2 regularization and token aggregation strategy Attention probe ~full-sequence hidden-state activations single attention-pooling head on activations from a chosen layer; sweeps over learning rate, weight decay, dropout, and number of heads SAE probe max-pooled hidden-state activations activations encoded through a pretrained sparse autoencoder, max-pooled across token positions; L1-regularized logistic regression on top-K most correlated features; sweeps over K and L2 regularization TF-IDF CoT text logistic regression over TF-IDF features extracted from the CoT text; sweeps over L2 regularization with cross-validation Zero-shot LLM monitor (binary) GPT-5.2 with reasoning reads the CoT and assigns a binary class label, given only a task description Few-shot LLM monitor (binary) GPT-5.2 with reasoning reads the CoT and assigns a binary class label, given a task description and a handful of labeled examples Zero-shot LLM monitor (confidence) GPT-5.2 with reasoning reads the CoT and scores the rollout 0–100, given only a task description Few-shot LLM monitor (confidence) GPT-5.2 with reasoning reads the CoT and scores the rollout 0–100, given a task description and a handful of labeled examples For all probe-based methods (linear, attention, and SAE), we run 50-trial Optuna hyperparameter sweeps, selecting the trial that maximizes g-mean² on the validation set. The best trial's threshold is also tuned on the validation set by sweeping over 99 evenly spaced values in [0.01, 0.99] and selecting the threshold that maximizes g-mean². This threshold is then frozen for both ID test and OOD evaluation — no re-tuning is performed OOD. We evaluate all methods in and out-of-distribution. The OOD regime is more interesting: methods that generalize across tasks and prompts without per-task supervision are far more useful in practice. Some tasks turn out t