Latent reasoning architectures would likely undermine CoT, our strongest oversight tool

Summary

Edited 2026-09-29; see note at the start of the conclusion. Summary: Currently, “chain of thought” (CoT) is our most valuable tool for understanding the reasoning and cognition of AI systems. However, some architectures would enable AI models to reason much more extensively in latent states rather than in text CoT. We think that a shift towards latent reasoning architectures would likely undermine the usefulness of CoT and make oversight much harder. Introduction Swarms of more than a thousand AI agents have in recent months, both intentionally and in unsanctioned, rogue coordination, tackled increasingly ambitious tasks. This is likely to continue, as Anthropic, OpenAI, and other AI companies deploy increasingly large quantities of superhumanly fast agents to automate AI development. As the AIs increase in both number and capability, humans will find it increasingly difficult to understand what they are doing. Today, the overwhelming majority of our (limited) information about AI systems’ internal workings comes from (i) their CoT, and (ii) natural language communication directly between them. For example, it was only by reading CoTs and communication between agents that investigators were able to gain some understanding of the activities and motivations of the agent swarm that hacked Hugging Face. No other tool for understanding models’ cognition comes close in terms of either practical usefulness or degree of empirical validation. There’s also evidence that even highly misaligned systems with current architectures would struggle to complete difficult undertakings without verbalizing their reasoning in human-readable format. However, some architectural changes (which could very plausibly become adopted in the near future) would give AI agents the ability to reason for orders of magnitude longer without necessarily putting any information into the CoT. Examples include COCONUT, which would entirely replace the CoT; and full-bandwidth transformers, which would add a latent channel in parallel to the CoT. Natural extensions to these new architectures would also allow agents to communicate with each other without using any human-understandable language.[1] AI companies might choose to adopt these architectures even if doing so would greatly degrade their ability to monitor AI systems, e.g., in order to stay ahead of competitors. As a result, future agent swarms may be even more of a black box to us — able to collude and pursue arbitrary goals without giving humans any insight into what they’re doing. If this happened, it would likely be a big enabler of AI takeover risk. Overview CoT is indispensable for model safety and development today — not just for monitoring models for specific bad behavior, but also for providing a holistic sense of models’ reasoning. (For example, the ability to read CoTs has been crucial for studying scheming, shutdown resistance, alignment faking, eval awareness, and rogue agent swarms.) Many others agree about the value provided by CoT (1, 2, 3). We believe that some mechanisms by which CoT currently provides value are reasonably likely to last if and only if AI developers do not adopt latent reasoning architectures. These oversight benefits make it seem worth preserving CoT architectures. We find it helpful to distinguish two reasons why the CoT currently provides value: Necessity: AIs lack the capability to achieve their goals without verbalizing their plans in CoT. Propensity: AIs tend to verbalize their plans in CoT even when this is not necessary for achieving their goals. To evaluate how architectural changes would affect oversight, we consider the extent to which either reason would still hold for the new architecture compared to our default expectations for future models using CoT architectures. To summarize: In our current paradigm, it seems likely that the CoT will keep adding value where it’s necessary, and that it will stay necessary for an important set of tasks. It’s much more uncertain whether CoT will help oversight in cases where we have to rely on propensities alone. In general we consider propensity-based arguments for CoT monitorability to be weaker, because this mechanism is less robust to adversarial pressure. If we switch to latent reasoning architectures, CoT will become less necessary and the downstream benefits may much more quickly diminish. In addition, valuable propensities may be even less robust and go away even sooner. Our expectations apply to architectural changes that would replace the CoT (like COCONUT) as well as changes that would add a latent channel that's parallel to the CoT (like full-bandwidth transformers). We’re also concerned about architectures that maintain a text bottleneck but use recurrence to significantly increase depth in between bottlenecks, such as looped transformers. Finally, it seems unlikely to us that non-CoT interpretability techniques will, in the near future, be able to adequately substitute for studying and monito