Quick Paper Review: "There Will Be a Scientific Theory of Deep Learning"

Summary

h/t Eric Michaud for sharing his paper with me. There’s a tradition of high-impact ML papers using short, punchy categorical sentences as their titles: Understanding Deep Learning Requires Rethinking Generalization, Attention is All You Need, Language Models Are Few Shot Learners, and so forth.  A new paper by Simon et al. seeks to expand on this tradition with not a present claim but a future tense, prophetic future sentence: “There Will Be a Scientific Theory of Deep Learning”.  There’s a lot of pessimism toward deep learning theory basically everywhere: the people building the AIs are pretty pessimistic, the academic AI researchers are as a general rule pessimistic (even people who used to do theory!), and with the exception of maybe 3-4 research groups, the independent AI safety ecosystem has long since given up on hoping for a theory to understand deep learning.  The paper is less of a neutral assessment of the evidence and more of a manifesto arguing for a particular theoretical deep learning research agenda. Given the overall sense of doom and gloom, its form makes sense: any less and it might not be enough to shine through the general sense of pessimism to all deep learning theory.  So what’s in the paper?  The authors start by introducing what they believe to be the new emerging theory of deep learning: “learning mechanics” (its name is a deliberate nod to physics theories such as statistical mechanics or quantum mechanics). In the authors’ words, learning mechanics is a theory that concerns itself with ” the dynamics of the training process”, studies them using “coarse aggregate statistics of learning”, and has the goal to generate “accurate average-case predictions”.  (In this sense, this is less a theory of deep learning as a whole and a theory that describes important aspects of deep learning. I’ll return to this later in this piece.) The authors lay out why such a theory is important. First, there’s the scientific reason: understanding the dynamics may help us better understand the nature of intelligence and the natural world. Second, there’s the practical, engineering reason: a clear characterization of learning dynamics would provide guidance for LLM training. Third, there’s the AI safety reason: understanding the systems better may help with regulation and AI governance, and it’s possible that learning dynamics may contribute to mech interp.  The authors then present five lines of evidence for why learning mechanics both exists, and is likely to become a “theory of deep learning”: There exist toy settings that we can analytically solve, that also yield insights that may transfer to large models in practice. Most of these results are from either deep linear networks or linearized versions of neural networks, though recently theoretical progress has been made on toy non-linear neural networks (e.g. 2 layer networks or attention-only models).  We can take the infinite width or infinite depth limit of neural networks, which sometimes yields interesting insights that can be applied for models in practice (the classic example is mu-parameterization). There are clear regularities between aggregate statistics of neural networks: the classic scaling laws that relate parameter count, dataset size, and loss, or various patterns in the weight dynamics, gradient alignment, or basin width over the course of training. While there aren’t many examples of theory allowing us to produce novel predictions of aggregate statistics, the fact that these clear regularities exist, and some theoretical progress has been made in explaining them, is a reason for hope.  We’ve made progress in terms of understanding and disentangling hyperparameters. Here lies perhaps the main concrete applications of deep learning theory: generating novel rules-of-thumb for scaling learning/initialization hyperparameters as you increase the amount of data or model parameters (again, mu-parameterization is the classic example). We’ve found universality in inductive biases, data structure, and representations. That is, it seems that different deep neural networks architectures seem to learn similar representations because many datasets also have similar properties. Again, while the theory is still nascent, the fact that these universals exist is reason for hope.  The authors then spend a small number of words outlining the relationship between learning mechanics and each of: classical learning theory, information theory, physics of deep learning, neuroscience, SLT/dev interp, and empirical science of deep learning. They then spend a much larger number of words outlining the connections between learning mechanics and mechanistic interpretability: learning mechanics may be able to help mech interp by formalizing core assumptions or explain how mechanisms arise during training, while mechanistic interpretability may be able to inspire phenomena to study with learning mechanics (as it has done in the past) Next, the authors respond to ar