Title: Sparsity and Symbols with Kolmogorov-Arnold Networks

Abstract: In this talk I’ll review Kolmogorov-Arnold nets, as well as new theory and applications related to sparsity and symbolic regression, respectively.  I’ll review essential results regarding KANs, show how sparsity masks relate deep nets and KANs, and how KANs can be utilized alongside multimodal language models for symbolic regression. Empirical results will necessitate a few slides, but the bulk will be chalk.

Title: Transformers and random walks: from language to random graphs

Abstract: The stunning capabilities of large language models give rise to many questions about how they work and how much more capable they can possibly get. One way to gain additional insight is via synthetic models of data with tunable complexity, which can capture the basic relevant structures of real data. In recent work we have focused on sequences obtained from random walks on graphs, hypergraphs, and hierarchical graphical structures. I will present some recent empirical results for work in progress regarding how transformers learn sequences arising from random walks on graphs. The focus will be on neural scaling laws, unexpected temperature-dependent effects, and sample complexity.

Title: LLMs, Reasoning, and the Future of Mathematical Sciences

Abstract: Over the last half decade, the mathematical capabilities of large language models (LLMs) have leapt from preschooler to undergraduate and now beyond. This talk reviews recent progress, and speculates as to what it will mean for the future of mathematical sciences if these trends continue.

Title: AlphaGeometry: a step toward automated math reasoning

Abstract: Last summer, Google DeepMind’s AI systems made headlines by achieving Silver Medal level performance on the notoriously challenging International Mathematical Olympiad (IMO) problems. For instance, AlphaGeometry 2, one of these remarkable systems, solved the geometry problem in a mere 19 seconds!

In this talk, we will delve into the inner workings of AlphaGeometry, exploring the innovative techniques that enable it to tackle intricate geometric puzzles. We will uncover how this AI system combines the power of neural networks with symbolic reasoning to discover elegant solutions.

Title: Ricci flow as a test for AI

Title: Gauss – towards autoformalization for the working mathematician

Abstract: In this talk we’ll highlight some recent formalization progress using a new agent – Gauss. We’ll outline a recent Lean proof of the Prime Number Theorem in strong form, completing a challenge set in January 2024 by Alex Kontorovich and Terry Tao. We hope Gauss will help assist working mathematicians, especially those who do not write formal code themselves.

Title: Spontaneous Kolmogorov-Arnold Geometry in Vanilla Fully-Connected Neural Networks

Abstract: The Kolmogorov-Arnold (KA) representation theorem constructs universal, but highly non-smooth inner functions (the first layer map) in a single (non-linear) hidden layer neural network. Such universal functions have a distinctive local geometry, a “texture,” which can be characterized by the inner function’s Jacobian, $J(\mathbf{x})$, as $\mathbf{x}$ varies over the data. It is natural to ask if this distinctive KA geometry emerges through conventional neural network optimization. We find that indeed KA geometry often does emerge through the process of training vanilla single hidden layer fully-connected neural networks (MLPs). We quantify KA geometry through the statistical properties of the exterior powers of $J(\mathbf{x})$: number of zero rows and various observables for the minor statistics of $J(\mathbf{x})$, which measure the scale and axis alignment of $J(\mathbf{x})$. This leads to a rough phase diagram in the space of function complexity and model hyperparameters where KA geometry occurs. The motivation is first to understand how neural networks organically learn to prepare input data for later downstream processing and, second, to learn enough about the emergence of KA geometry to accelerate learning through a timely intervention in network hyperparameters. This research is the “flip side” of KA-Networks (KANs). We do not engineer KA into the neural network, but rather watch KA emerge in shallow MLPs.

Title:

Abstract: We introduce a method of topological analysis on spiking correlation networks in neurological systems. This method explores the neural manifold as in the manifold hypothesis, which posits that information is often represented by a lower-dimensional manifold embedded in a higher-dimensional space. After collecting neuron activity from human and mouse organoids using a micro-electrode array, we extract connectivity using pairwise spike-timing time correlations, which are optimized for time delays introduced by synaptic delays. We then look at network topology to identify emergent structures and compare the results to two randomized models – constrained randomization and bootstrapping across datasets. In histograms of the persistence of topological features, we see that the features from the original dataset consistently exceed the variability of the null distributions, suggesting that the observed topological features reflect significant correlation patterns in the data rather than random fluctuations. In a study of network resiliency, we found that random removal of 10 % of nodes still yielded a network with a lesser but still significant number of topological features in the homology group H1 (counts 2-dimensional voids in the dataset) above the variability of our constrained randomization model; however, targeted removal of nodes in H1 features resulted in rapid topological collapse, indicating that the H1 cycles in these brain organoid networks are fragile and highly sensitive to perturbations. By applying topological analysis to neural data, we offer a new complementary framework to standard methods for understanding information processing across a variety of complex neural systems.

Title: The Shape of Math to Come

Abstract: We will discuss some ongoing experiments that may have meaningful impact on what working in research mathematics might look like in a decade (if not sooner).

Title: FrontierMath to Infinity

Abstract: I will discuss FrontierMath, a mathematical problem solving benchmark I developed over the past year, including its design philosophy and what we’ve learned about AI’s trajectory from it. I will then look much further out, speculate about what a “perfectly efficient” mathematical intelligence should be capable of, and discuss how high-ceiling math capability metrics can illuminate the path towards that ideal.

Title:Demystifying the over-parametrization of neural networks

Abstract: I will show how to estimate the dimensionality of neural encodings (learned weight structures) to assess how many parameters are effectively used by a neural network. I will then show how their scaling properties provide us with fundamental exponents on the learning process of a given task. I will comment on connections to thermodynamics.

Title: Math for AI and AI for Math

Abstract: I will briefly discuss two DARPA programs aiming to deepen connections between mathematics and AI, specifically through geometric and symbolic perspectives. The first aims for mathematical foundations for understanding the behavior and performance of modern AI systems such as Large Language Models and Diffusion models. The second aims to develop AI for pure mathematics through an understanding of abstraction, decomposition, and formalization. I will close with some thoughts on the coming convergence between AI and math.

Title: How to think about the shape of mathematics

Followed by group discussion