New Technologies in Mathematics Seminar

https://youtu.be/ztk5TPYTKZA New Technologies in Mathematics Seminar Speaker: Jimmy Ba, University of Toronto Title: How to steer foundation models? Abstract: By conditioning on natural language instructions, foundation models and large language models (LLMs) have displayed impressive capabilities as general-purpose computers. However, task performance depends significantly on the quality of the prompt used to steer the model. […]

https://youtu.be/UID_oXuN-0Y New Technologies in Mathematics Seminar Speaker: Timo Schick, Meta AI Title: Toolformer: Language Models Can Teach Themselves to Use Tools Abstract: Language models exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where […]

https://youtu.be/5LXiQUsnHrI New Technologies in Mathematics Seminar Speaker: Dmitry Krotov, IBM Research - Cambridge Title: Modern Hopfield Networks for Novel Transformer Architectures Abstract: Modern Hopfield Networks or Dense Associative Memories are recurrent neural networks with fixed point attractor states that are described by an energy function. In contrast to conventional Hopfield Networks, which were popular in […]

https://youtu.be/wTQH6mRDXhw New Technologies in Mathematics Seminar Speaker: Ronen Eldan, Microsoft Research Title: The TinyStories Dataset: How Small Can Language Models Be And Still Speak Coherent Abstract: While generative language models exhibit powerful capabilities at large scale, when either the model or the number of training steps is too small, they struggle to produce coherent and fluent […]

https://youtu.be/Sc6k06wVX3s New Technologies in Mathematics Seminar Speaker: François Charton, Meta AI Title:  Transformers for maths, and maths for transformers Abstract: Transformers can be trained to solve problems of mathematics. I present two recent applications, in mathematics and physics: predicting integer sequences, and discovering the properties of scattering amplitudes in a close relative of Quantum ChromoDynamics. Problems of […]

https://youtu.be/u-pkmdkQoMU New Technologies in Mathematics Seminar Speaker: Alex Gu, MIT Dept. of EE&CS Title: LeanDojo: Theorem Proving with Retrieval-Augmented Language Models Abstract: Large language models (LLMs) have shown promise in proving formal theorems using proof assistants such as Lean. However, existing methods are difficult to reproduce or build on, due to private code, data, and large compute […]

https://youtu.be/M25cbX5do8Y New Technologies in Mathematics Seminar Speaker: Yuanzhi Li, CMU Dept. of Machine Learning and Microsoft Research Title: Physics of Language Models: Knowledge Storage, Extraction, and Manipulation Abstract: Large language models (LLMs) can memorize a massive amount of knowledge during pre-training, but can they effectively use this knowledge at inference time? In this work, we show several striking […]

https://youtu.be/bRHb-MVExJ4 New Technologies in Mathematics Seminar Speaker: Sean Welleck, CMU, Language Technologies Institute Title: Llemma: an open language model for mathematics Abstract: We present Llemma: 7 billion and 34 billion parameter language models for mathematics. The Llemma models are initialized with Code Llama weights, then trained on the Proof-Pile II, a 55 billion token dataset of […]

https://youtu.be/a2S_-pl6onM New Technologies in Mathematics Seminar Speaker: Gabriel Poesia, Dept. of Computer Science, Stanford University Title: Peano: Learning Formal Mathematical Reasoning Without Human Data Abstract: Peano is a theorem proving environment in which a computational agent can start tabula rasa in a new domain, learn to solve problems through curiosity-driven exploration, and create its own higher level […]

https://youtu.be/JYt-ldZ3DqM New Technologies in Mathematics Seminar Speaker: Abhishek Panigrahi, Dept. of Computer Science, Princeton University Title: On the Power of Forward pass through Transformer Architectures Abstract: Highly trained transformers are capable of interesting computations as they infer for an input. The exact mechanism that these models use during forward passes is an interesting area of […]

https://youtu.be/oiOpudgC0J4 New Technologies in Mathematics Seminar Speaker: Heather Macbeth, Fordham University Title: Approaches to the formalization of differential geometry Abstract: In the last five years, there has been early work on the computer formalization of differential geometry. I will survey the projects I am aware of. I will also describe two projects of my own, […]