Towards surfacing model algorithms with meta-tokens in the J-Space

Summary

TL;DR We used J-lens on Qwen3.6-27B to find “meta-tokens”: tokens that surface non-obvious computation in the model. When the model reads ambiguous text, 什么意思 ("what does this mean") fires in the J-space, and steering it away makes the model answer "a boiled egg every morning is hard to beat" with nutrition facts instead of catching the pun. On LCM problems, "gcd" fires, pointing at the product/GCD algorithm: swapping the GCD vector from 9 to 3 makes the model change its answer for the LCM of 27 and 90 from 270 to 810. Additionally, before the model hedges, 大概率 ("most likely") fires, and suppressing it makes the model commit to a single option (for e.g. "There are several logical places John could have gone" turns into "he went to the stationery store)." While meta-tokens are hard to find with J-lens’ single token constraint, future methods for multi-token J-lens open the possibility for valuable insights into the model’s internal algorithms. Motivation One important goal in interpretability is to surface the variables the model uses in its computations. Another is to surface the algorithms that give rise to these variables. We can call the former variable interpretability and the latter algorithm interpretability (see e.g. Engels et al). Recent work from Anthropic introduced an interpretability technique called the J-lens, which seems to allow us to access a cognitive space in the model. J-lens is a technique for interpreting intermediate variables in the residual stream, so initially its applications to algorithm interpretability seem highly indirect (i.e. looking at which intermediates are used can narrow down algorithms). Expanding on our preliminary findings, we find evidence that J-lens can in some cases more directly indicate the algorithm used by the model. What are “meta-tokens”? We loosely define meta-tokens as tokens that appear in J-lens readouts and give non-obvious evidence of the type of processing/algorithm occurring in the model. The token vectors are not required to be causal, but should at least correspond to some causal concept in the model. These tokens generally show up in the “workspace layers” (a band ~30-90% of the way through the model). In Qwen, we find "meta-tokens" to often be Chinese tokens, which are much more information dense per character than English characters. Since the J-Lens can only find concepts corresponding to a single-token, it is able to find more interesting concepts in Qwen. Why are we writing this post? Proof of concept that J-lens can help us surface nontrivial computations in the model beyond intermediate variables Catalog interesting case studies of interesting meta-tokens, and as interesting test cases for multi-token J-Lens attempts (the natural next step for the technique). Replication We replicate and fit a J-lens for Qwen 3.6-27B (64 layers), taking the Jacobian with respect to the penultimate layer, averaging over 25 sequences of 128 tokens from the Pile (NeelNanda/pile-10k) mainly, while some experiments use wikitext. We skipped the first four tokens per prompt during averaging since they had an anomalously high norm. We note that as this is a different and weaker model so some results should differ. Anthropic's ablations show that averaging over n=10 prompts is nearly as good as their n=1000, and we find n=25 in our case to work well enough. We report key experimental replications of our J-lens in the Appendix. Case studies Interpretative meta-tokens We found four interpretative meta-tokens: 什么意思 (what meaning), 是什么意思 (what does it mean), 这句话 (this sentence), 是何含义 (what does it mean). These seem to have similar functions so we will study them collectively. These appear on sentences that are ambiguous or have a non-obvious context (like poems, crossword hints, jokes, tweets, etc.), and they have a causal role on the model’s ability to disambiguate the context of a prompt.[1] We first observed these meta-tokens on ambiguous poetry prompts, like “the drummer boy kept the marching column in line,\n”, particularly on the new line token. When looking at the J-lens readout, the meta-tokens (green) appear right before the model seems to recognize the context (“poem”/“song” in orange). As further evidence that these meta-tokens are related to disambiguation, when we clarify the context of the prompt with a prefix like “A poem:”, the meta-tokens largely do not show up and are replaced by more genre tokens. Eval setup We test if the interpretative meta-tokens play a causal role in the model figuring out the context of a mildly ambiguous sentence, like a poem or a joke. We negatively steer the J-lens directions for the meta-tokens. We construct three types of ambiguous/non-obvious settings (puns, rhymes, and implied wordplay) and see how the model’s response changes when steering with meta-token vectors and random vectors as controls. Steering details: Steered at 1) all workspace layers, and 2) all layers where the meta-tokens were salient (defined by t