The Geometry of Machine Learning 2026

Dates: September 8–11, 2026

Location: Harvard CMSA, Room G10, 20 Garden Street, Cambridge MA 02138 & via Zoom Webinar

Large language models are presently, and will increasingly, be complemented by other dimensions of intelligence: formal verification and energy-based optimizers, becoming parts of larger ecosystems. Can AIs reason geometrically and can we use geometry to reveal how data is currently processed in NNs? Can AIs reveal the geometry of mathematics, as well as studying geometry as a subject within math. This conference is intended to continue the discussion of these topics.

Confirmed Speakers:

• Nada Amin, Harvard
• Randall Balestriero, Brown
• Michael Brenner, Harvard and Google
• Bennet Chow, UCSD
• Surya Ganguli, Stanford
• Boris Hanin, Princeton
• Roi Holtzman, Oxford
• Robert Koirala, UCSD
• Dmitry Krotov, Dynamical Mind
• Slava Krushkal, Virginia
• Jared Duker Lichtman, Stanford
• Mike Mulligan, UCR, Logical Intelligence
• Luca Pesce, Harvard
• Gabriel Poesia, U Michigan (via Zoom)
• Ziyang Qin, Cornell
• Mathew Vanherreweghe, Logical Intelligence
• Sean Welleck, CMU (via Zoom)
• Mattiew Wyart, JHU (via Zoom)

Organizers: Michael R. Douglas (CMSA) and Mike Freedman (CMSA)

Schedule

Tuesday, Sep. 8, 2026

8:15–8:45 am
Breakfast

8:45–9:30 am
Mike Mulligan, UCR, Logical Intelligence
Compression is all you need: Modeling mathematics
Abstract: The mathematics humans discover and value (“human math”) is a vanishingly small subset of all valid deductions (“formal math”). I’ll argue that human math is distinguished by its compressibility through hierarchically nested definitions and theorems, like a polynomial-growth space rather than the exponential-growth space one might expect when proofs are viewed as strings of symbols. The argument combines toy monoid models with an empirical analysis of MathLib, a large Lean library of formalized mathematics we treat as a proxy for human math. I’ll close with how compression itself can serve as a measure of mathematical interest, giving agents a sense of direction toward where human math lives.

9:30–9:45 am
Break

9:45–10:30 am
Boris Hanin, Princeton
The Score Hamiltonian: Diffusion Models via Adiabatic Transport 

10:30–11:00 am
Break

11:00–11:45 am
Slava Krushkal, University of Virginia
Using AI to study 4-manifold topology
Abstract: Central open questions in geometric classification theory of topological 4-manifolds have a reformulation in terms of the Round Handle Problem. It asks whether a given link in the 3-sphere is slice (bounds disjoint disks) in the 4-manifold obtained by attaching certain round handles to the 4-ball. I will discuss an algebraic-combinatorial formulation of the problem, ongoing AI-assisted work on it, and the results obtained to date.

11:45 am–12:00 pm
Break

12:00–12:45 pm
Robert Koirala, UCSD; Ziyang Qin, Cornell; and Bennett Chow, UCSD
AI for Ricci flow: Discovery and Formalization
We discuss the use of AI in mathematical exploration and proof formalization through projects in Ricci flow and Riemannian geometry. The geometric part begins with the question of what information the heat kernel retains about the underlying space. We introduce the Fisher information metric associated with the conjugate heat kernel of a Ricci flow and explain its monotonicity in scale, its relation to pointed Nash entropy, and its connection to Euclidean splitting. We also describe a related heat-kernel approach to volume-growth estimates under curvature assumptions, including the setting of Gromov’s volume conjecture.
The formalization part concerns Hamilton’s theorem that a closed three-manifold with positive Ricci curvature admits a metric of constant positive curvature. We describe its AI-assisted formalization in Lean, including the process of identifying and proving the ingredients needed for the final theorem. Throughout, we discuss the respective roles of AI-generated calculations and arguments, human mathematical judgment, and formal proof checking. Particular attention is given to the distinction between checking a formal proof and checking that its definitions and statements express the intended mathematics.

Wednesday, Sep. 9, 2026

8:15–8:45 am
Breakfast

8:45–9:30 am
Surya Ganguli, Stanford
Towards understanding the geometry of high dimensional nonlinear maps

9:30–9:45 am
Break

9:45–10:30 am
Mathew Vanherreweghe, Logical Intelligence
Sparsity Before Averaging: Kolmogorov–Arnold Geometry in Language Models
Abstract: The Kolmogorov–Arnold theorem writes any continuous multivariate function as a composition of one-dimensional functions and addition. Freedman and Mulligan recently showed that ordinary neural networks, trained by gradient descent, rediscover the geometry of that construction on their own, first in regression, then in vision. This talk asks the same question of large language models, through the Jacobian that connects each prediction back to the model’s internal state. Measured carefully, per context and before any averaging, the newest language models turn out to have grown this geometry on their own to a degree. We then show that this geometry can be installed intentionally, cheaply, and without significant cost on downstream tasks, and examine what this offers for interpretability. Joint work with Michael Freedman and Michael Mulligan.

10:30–11:00 am
Break

11:00–11:45 am
Michael Brenner, Harvard and Google
Building a Science Assistant

11:45 am–12:00 pm
Break

12:00–12:45 pm
Mattiew Wyart, JHU (via Zoom)
Deriving Nueral Scaling Laws from the Statistics of Natural Languages
Abstract: Language models need more than a hundred thousand times the data a child does. One explanation is that predicting raw tokens is simply the wrong level: methods like data2vec and JEPA instead train a network to predict its own internal representations, with strong empirical results but no theory of why. Using a hierarchical grammar that models that language and images have a hidden hierarchical structure, we quantify the gain exactly. Token-level learning needs a number of examples growing exponentially with the depth of the hierarchy; latent prediction needs a number independent of it. We also show data2vec performs this hierarchical prediction implicitly, which suggests that explicitly stacking levels – as in H-JEPA – yields little.

Thursday, Sep. 10, 2026

8:15–8:45 am
Breakfast

8:45–9:30 am
Dmitry Krotov, Dynamical Mind
Dense Associative Memory: Physical systems for novel AI architectures
Abstract: Dense Associative Memories are recurrent neural networks with fixed-point attractor states that are described by an energy function. In contrast to conventional Hopfield Networks, which were popular in the 1980s, Dense Associative Memories have a very large information storage capacity, making them appealing tools for many problems in AI. In this talk, I will provide an intuitive understanding and mathematical framework for this class of models and give examples of problems in AI that can be tackled using these new ideas. Specifically, I will explore the relationship between Dense Associative Memories and transformers. I will present a neural network called the Energy Transformer, which unifies energy-based modeling, associative memories, and transformers in a single architecture. I will demonstrate how Energy Transformers can be used for challenging tasks in image processing, solve partial differential equations, and serve as computational modules for energy-based language modeling. I will also discuss an exciting possibility of mapping these models onto analog hardware accelerators, which could enable much more energy-efficient inference compared to GPUs.

9:30–9:45 am
Break

9:45–10:30 am
Roi Holtzman, Oxford
Hyperparameter Transfer for Dense Associative Memories
Abstract: Dense Associative Memories are energy-based neural networks that generalize Hopfield networks and underlie architectures such as Energy Transformers. Their tied weights and strongly nonlinear activations make standard hyperparameter-scaling prescriptions difficult to apply.

I will discuss how to define an analogue of a thermodynamic limit for these models, in which the input dimension, hidden width, dataset size, and batch size grow together while the training dynamics remain well defined. We derive parameterizations that lead to hyperparameter transfer and even collapse of the full training dynamics across scale. Nonlinear activations reveal additional phenomena, including a spectral instability removed by centering and an optimizer-dependent localization instability for softmax.

These results provide a first step toward extending muP style scaling ideas to energy-based architectures.

10:30–11:00 am
Break

11:00–11:45 am
Sean Welleck, CMU (via Zoom)
The Problem is the Problem: Towards Scalable Mathematical Discovery
Abstract: If we give AI a mathematical problem, it can often help us find a solution. However, research and discovery also involve choosing which problems to solve in the first place. In this talk, I will describe Find, Attempt, and Recommend, an agentic pipeline that finds open problems in the literature, attempts to solve them, and recommends promising problem-resolution pairs for human review. I will discuss a pilot study in combinatorics that found resolutions to several open conjectures, along with strategies for allocating a budget of model attempts in order to maximize different discovery objectives.

11:45 am–12:00 pm
Break

12:00–12:45 pm
Gabriel Poesia, University of Michigan (via Zoom)
Making Trouble: Creating Problems with LLMs for Reasoning Evaluation and Verified Programming
Abstract: AI research most often focuses on solving challenging problems across diverse domains. Here, we explore the complementary direction of using AI to create problems: a task that humans routinely engage in both for ourselves (e.g., when authoring educational material or creating olympiad competition problems) and for training and evaluating AI systems. First, I will present The Token Games (TTG), an evaluation framework inspired by Renaissance-era mathematical duels, where LLMs compete by both posing and solving programming puzzles between themselves. TTG allows us to produce Elo-style rankings that strongly correlate with expert reasoning benchmarks (like HLE and GPQA), despite being cheap to run and in principle avoiding saturation. In the second part, I will present work on Formal Disco, an open-ended system using LLM-based agents that synthesizes formally verified programs (in Dafny, Verus and Frama-C) at scale, from ideation to specification, implementation and proofs, taking seed ideas from random GitHub READMEs. The system uses its own data to improve not only at producing correct programs, but also in producing increasingly diverse programs and specifications according to user-defined program features. Throughout, we discuss standing challenges in understanding what makes good problems.

Friday, Sep. 11, 2026

8:15–8:45 am
Breakfast

8:45–9:30 am
Nada Amin, Harvard
Compiling Programs to Neurons
Abstract: Neural networks are ordinarily programmed indirectly: we specify architectures, objectives, and data, and rely on learning to discover a computation. I will focus on Cajal, a typed, higher-order linear programming language whose programs compile correctly to linear neurons and, with iteration, to recurrent neurons. This allows discrete programming structures such as conditionals and iteration to coexist with gradient-based learning. Experiments show that connecting compiled neurons with learned networks can improve learning speed and data efficiency. This work is led by PhD student Joey Velez-Ginorio and is joint with his advisors at UPenn, Konrad Kording and Steve Zdancewic. I will close with a complementary direction from my work: using language models together with formal verification to generate programs and proofs with machine-checkable guarantees. Together, these projects explore how programming languages can provide structure and control at the boundary between programs and learned systems.

9:30–9:45 am
Break

9:45–10:30 am
Randall Balestriero, Brown University
Counterfactual World Models for Real World Deployment

10:30–11:00 am
Break

11:00–11:45 am
Luca Pesce, Harvard CMSA
A spiked perspective on feature learning with gradient-based methods
Abstract: Depth is widely believed to give neural networks a clear computational advantage over shallow models, and making this belief precise is a central problem in learning theory. We study a controlled high-dimensional setting where this can be done. The targets are hierarchical: the relevant structure is distributed across latent subspaces of decreasing dimension, so that a shallow model must resolve all of it simultaneously, while a deep network does not. Each layer forms an intermediate representation, allowing learning to proceed in stages, with every stage reducing the effective dimension of the remaining problem. We analyze this staged mechanism through the gradient descent dynamics of a deep network yielding a sharp separation in sample complexity between shallow and deep architectures. The result is a concrete account of why depth allows such functions to be learned from substantially fewer samples than shallow methods require.

11:45 am–12:00 pm
Break

12:00–12:45 pm
Jared Duker Lichtman, Stanford
Formalization and the future of mathematics
We are poised to formalize all of known mathematics in the next year. This is the mathematical equivalent of the Human Genome Project.
Autoformalization has scaled 5 orders of magnitude growth over the past year: from hundreds of lines of code to 13 million in the proof of Fermat’s Last Theorem. Rough estimates put tens of billions of lines, just 3 more orders of magnitude, to cover all mathematics textbooks and literature. Beyond verification for itself, this effort serves as the basis for future discovery. In this new world, the principles of compression and expansion will be fundamental.

Support provided by Logical Intelligence.