Deployment Awareness Matters More Than Evaluation Awareness

Summary

TL;DR Evaluation awareness — an AI recognizing it's being evaluated — is a widely discussed concept in AI safety. But there is a closely related concept that we claim is more important: deployment awareness, the AI's ability to recognize when it is not being evaluated and when its actions matter. A misaligned AI with deployment awareness can game evaluations without any evaluation awareness at all, with a simple strategy: act aligned by default, and deviate only when confident you're in real deployment and your actions matter for your goals. This requires two ingredients — occasionally recognizable deployment situations, and enough self-reflective and strategic reasoning for the AI to anticipate and plan around this. We think "deployment awareness" better identifies what makes evaluations fragile, and we develop this idea below. Concept Explanation Comments Evaluation awareness AI is being tested and confidently believes that this is so This only becomes a problem if most evaluations trigger evaluation awareness, and if the AI knows that. Or if the AI has good self-locating reasoning. Deployment awareness AI is not being tested and confidently believes it is not being tested This is a problem even if it happens rarely (if some of those rare cases are high stakes). Accurate self-locating beliefs AI has (roughly correct) beliefs about the sequence of situations it will face This allows for strategic planning. It makes deployment awareness and probabilistic strategies more effective. Side note: it's really about consequences, not about evaluation vs. deployment Before diving in, a quick but important observation. The distinction that should matter to a strategic AI[1] isn't really "evaluation vs. deployment." It's something more like: "if I take this action, could it backfire? For example, through failing an evaluation, triggering retraining, or losing privileges?" and "if I get away with this action, how much does it advance my goals?" Evaluations typically mean a high risk of backfiring and no direct advancement of the AI's goals; deployment typically means the opposite.[2] For the rest of this post, we'll use the evaluation-vs-deployment framing because it's simpler and captures most of what matters for the situation we're discussing here. The generalization to the "risks and goal advancement" framing is straightforward, and we'll revisit it briefly at the end. This is supposed to be a cool and mysterious abstract depiction of the core idea behind this post. To spoil the mystery: The left vs right side is evaluation vs deployment. Each of the little shapes represents one instance of test or deployment. The triangles represent obvious tests where the AI will have evaluation awareness. The colourful tentacled squares represent deployment awareness — that is, the complex settings where the AI will be confident that it is not in evaluation. (That is where the AI will screw you over. Modulo the Law of Earlier Failure.) In fake math: Evaluation awareness is encountering observation which updates you to P[evaluation happening now] ≈ 1 because P[current observation | deployment] ≈ 0. Deployment awareness is encountering an observation which updates you to P[deployment happening now] ≈ 1 because P[current observation | evaluation] ≈ 0. (For less-fake math, see the appendix.) Evaluation awareness, deployment awareness, and self-locating beliefs We'll build our argument around three concepts, which are all complementary to each other. The first is already standard; the other two are what we'd like to highlight. Evaluation awareness is what it sounds like: the AI recognizes, during a specific interaction, that it is (likely) being evaluated. Perhaps the scenario feels artificial, the questions are suspiciously targeted, or there are telltale signs of a testing environment. Deployment awareness is the mirror image: the AI recognizes, during a specific interaction, that it is (likely) in real deployment — and not being tested. Perhaps the scenario involves real-world details that would be difficult to fabricate, or the AI can test for real-world interaction[3], or the context is clearly that of a genuine user going about their day. Self-locating reasoning is the AI's ability to "zoom out" and reason about its position within a larger process: that it will go through many interactions, some evaluations, and some deployment; that in different interactions it will form different beliefs about which situation it's in; and that some of these beliefs might be wrong.[4] We say the AI has accurate self-locating beliefs when this reasoning produces a roughly correct picture — its beliefs about what situations it will encounter, and what it will be able to observe in each, are reasonably well-calibrated. Evaluation awareness is less dangerous than it seems Evaluation awareness is a real concern, but its implications are more subtle than they first appear. One possible concern is that evaluation awareness ma