On August 31-Sep 1, 2023 the CMSA will host the ninth annual Conference on Big Data. The Big Data Conference features speakers from the Harvard community as well as scholars from across the globe, with talks focusing on computer science, statistics, math and physics, and economics.
Location: Harvard University Science Center Hall D & via Zoom.
Title: From Network Medicine to the Foodome: The Dark Matter of Nutrition
Abstract: A disease is rarely a consequence of an abnormality in a single gene but reflects perturbations to the complex intracellular network. Network medicine offer a platform to explore systematically not only the molecular complexity of a particular disease, leading to the identification of disease modules and pathways, but also the molecular relationships between apparently distinct (patho) phenotypes. As an application, I will explore how we use network medicine to uncover the role individual food molecules in our health. Indeed, our current understanding of how diet affects our health is limited to the role of 150 key nutritional components systematically tracked by the USDA and other national databases in all foods. Yet, these nutritional components represent only a tiny fraction of the over 135,000 distinct, definable biochemicals present in our food. While many of these biochemicals have documented effects on health, they remain unquantified in any systematic fashion across different individual foods. Their invisibility to experimental, clinical, and epidemiological studies defines them as the ‘Dark Matter of Nutrition.’ I will speak about our efforts to develop a high-resolution library of this nutritional dark matter, and efforts to understand the role of these molecules on health, opening novel avenues by which to understand, avoid, and control disease.
Title: Differentially Private Algorithms for Statistical Estimation Problems
Abstract: Differential privacy (DP) is widely regarded as a gold standard for privacy-preserving computation over users’ data. It is a parameterized notion of database privacy that gives a rigorous worst-case bound on the information that can be learned about any one individual from the result of a data analysis task. Algorithmically it is achieved by injecting carefully calibrated randomness into the analysis to balance privacy protections with accuracy of the results. In this talk, we will survey recent developments in the development of DP algorithms for three important statistical problems, namely online learning with bandit feedback, causal interference, and learning from imbalanced data. For the first problem, we will show that Thompson sampling — a standard bandit algorithm developed in the 1930s — already satisfies DP due to the inherent randomness of the algorithm. For the second problem of causal inference and counterfactual estimation, we develop the first DP algorithms for synthetic control, which has been used non-privately for this task for decades. Finally, for the problem of imbalanced learning, where one class is severely underrepresented in the training data, we show that combining existing techniques such as minority oversampling perform very poorly when applied as pre-processing before a DP learning algorithm; instead we propose novel approaches for privately generating synthetic minority points.
Based on joint works with Marco Avella Medina, Vishal Misra, Yuliia Lut, Tingting Ou, Saeyoung Rho, and Ethan Turok.
Title: To split or not to split that is the question: From cross validation to debiased machine learning
Abstract: Data splitting is a ubiquitous method in statistics with examples ranging from cross-validation to cross-fitting. However, despite its prevalence, theoretical guidance regarding its use is still lacking. In this talk, we will explore two examples and establish an asymptotic theory for it. In the first part of this talk, we study the cross-validation method, a ubiquitous method for risk estimation, and establish its asymptotic properties for a large class of models and with an arbitrary number of folds. Under stability conditions, we establish a central limit theorem and Berry-Esseen bounds for the cross-validated risk, which enable us to compute asymptotically accurate confidence intervals. Using our results, we study the statistical speed-up offered by cross-validation compared to a train-test split procedure. We reveal some surprising behavior of the cross-validated risk and establish the statistically optimal choice for the number of folds. In the second part of this talk, we study the role of cross-fitting in the generalized method of moments with moments that also depend on some auxiliary functions. Recent lines of work show how one can use generic machine learning estimators for these auxiliary problems, while maintaining asymptotic normality and root-n consistency of the target parameter of interest. The literature typically requires that these auxiliary problems are fitted on a separate sample or in a cross-fitting manner. We show that when these auxiliary estimation algorithms satisfy natural leave-one-out stability properties, then sample splitting is not required. This allows for sample reuse, which can be beneficial in moderately sized sample regimes.
Abstract: Linear dynamical systems are the canonical model for time series data. They have wide-ranging applications and there is a vast literature on learning their parameters from input-output sequences. Moreover they have received renewed interest because of their connections to recurrent neural networks. But there are wide gaps in our understanding. Existing works have only asymptotic guarantees or else make restrictive assumptions, e.g. that preclude having any long-range correlations. In this work, we give a new algorithm based on the method of moments that is computationally efficient and works under essentially minimal assumptions. Our work points to several missed connections, whereby tools from theoretical machine learning including tensor methods, can be used in non-stationary settings.
Title: Algorithmic Thresholds for Spherical Spin Glasses
Abstract: High-dimensional optimization plays a crucial role in modern statistics and machine learning. I will present recent progress on non-convex optimization problems with random objectives, focusing on the spherical p-spin glass. This model is related to spiked tensor estimation and has been studied in probability and physics for decades. We will see that a natural class of “stable” optimization algorithms gets stuck at an algorithmic threshold related to geometric properties of the landscape. The algorithmic threshold value is efficiently attained via Langevin dynamics or by a second-order ascent method of Subag. Much of this picture extends to other models, such as random constraint satisfaction problems at high clause density.
Title: What Learning Algorithm is In-Context Learning?
Abstract: Neural sequence models, especially transformers, exhibit a remarkable capacity for “in-context” learning. They can construct new predictors from sequences of labeled examples (x,f(x)) presented in the input without further parameter updates. I’ll present recent findings suggesting that transformer-based in-context learners implement standard learning algorithms implicitly, by encoding smaller models in their activations, and updating these implicit models as new examples appear in the context, using in-context linear regression as a model problem. First, I’ll show by construction that transformers can implement learning algorithms for linear models based on gradient descent and closed-form ridge regression. Second, I’ll show that trained in-context learners closely match the predictors computed by gradient descent, ridge regression, and exact least-squares regression, transitioning between different predictors as transformer depth and dataset noise vary, and converging to Bayesian estimators for large widths and depths. Finally, we present preliminary evidence that in-context learners share algorithmic features with these predictors: learners’ late layers non-linearly encode weight vectors and moment matrices. These results suggest that in-context learning is understandable in algorithmic terms, and that (at least in the linear case) learners may rediscover standard estimation algorithms. This work is joint with Ekin Akyürek at MIT, and Dale Schuurmans, Tengyu Ma and Denny Zhou at Stanford.
Abstract: Rapidly advancing deep distributional modeling techniques offer a number of opportunities for complex generative tasks, from natural sciences such as molecules and materials to engineering. I will discuss generative approaches inspired from physical processes including diffusion models and more recent electrostatic models (Poisson flow), and how they relate to each other in terms of embedding dimension. From the point of view of applications, I will highlight our recent work on SE(3) invariant distributional modeling over backbone 3D structures with ability to generate designable monomers without relying on pre-trained protein structure prediction methods as well as state of the art image generation capabilities (Poisson flow). Time permitting, I will also discuss recent analysis of efficiency of sample generation in such models.
Abstract: Understanding biological and natural systems requires modeling data with underlying geometric relationships across scales and modalities such as biological sequences, chemical constraints, and graphs of 3D spatial or biological interactions. I will discuss unique challenges for learning from multimodal datasets that are due to varying inductive biases across modalities and the potential absence of explicit graphs in the input. I will describe a framework for structure-inducing pretraining that allows for a comprehensive study of how relational structure can be induced in pretrained language models. We use the framework to explore new graph pretraining objectives that impose relational structure in the induced latent spaces—i.e., pretraining objectives that explicitly impose structural constraints on the distance or geometry of pretrained models. Applications in genomic medicine and therapeutic science will be discussed. These include TxGNN, an AI model enabling zero-shot prediction of therapeutic use across over 17,000 diseases, and PINNACLE, a contextual graph AI model dynamically adjusting its outputs to contexts in which it operates. PINNACLE enhances 3D protein structure representations and predicts the effects of drugs at single-cell resolution.
Abstract: Uncertainty quantification for prediction is an intriguing problem with significant applications in various fields, such as biomedical science, economic studies, and weather forecasts. Numerous methods are available for constructing prediction intervals, such as quantile regression and conformal predictions, among others. Nevertheless, model misspecification (especially in high-dimension) or sub-optimal constructions can frequently result in biased or unnecessarily-wide prediction intervals. In this work, we propose a novel and widely applicable technique for aggregating multiple prediction intervals to minimize the average width of the prediction band along with coverage guarantee, called Universally Trainable Optimal Predictive Intervals Aggregation (UTOPIA). The method also allows us to directly construct predictive bands based on elementary basis functions. Our approach is based on linear or convex programming which is easy to implement. All of our proposed methodologies are supported by theoretical guarantees on the coverage probability and optimal average length, which are detailed in this paper. The effectiveness of our approach is convincingly demonstrated by applying it to synthetic data and two real datasets on finance and macroeconomics. (Joint work Jiawei Ge and Debarghya Mukherjee).
Title: Efficient OCR for Building a Diverse Digital History
Abstract: Many users consult digital archives daily, but the information they can access is unrepresentative of the diversity of documentary history. The sequence-to-sequence architecture typically used for optical character recognition (OCR) – which jointly learns a vision and language model – is poorly extensible to low-resource document collections, as learning a language-vision model requires extensive labeled sequences and compute. This study models OCR as a character-level image retrieval problem, using a contrastively trained vision encoder. Because the model only learns characters’ visual features, it is more sample-efficient and extensible than existing architectures, enabling accurate OCR in settings where existing solutions fail. Crucially, it opens new avenues for community engagement in making digital history more representative of documentary history.
Harvard Science Center, 1 Oxford Street, Cambridge MA 02138
Probability Seminar
Speaker: Marius Lemm, University of Tuebingen
Title: Light cones for open quantum systems
Abstract: We consider non-relativistic Markovian open quantum dynamics in continuous space. We show that, up to small probability tails, the supports of quantum states propagate with finite speed in any finite-energy subspace. More precisely, if the initial quantum state is localized in space, then any finite-energy part of the solution of the von Neumann-Lindblad equation is approximately localized inside an energy-dependent light cone. We also obtain an explicit upper bound on the slope of this light cone (i.e., on the maximal speed). The general method can be used to derive propagation bounds for a variety of other quantum systems including Lieb-Robinson bounds for lattice bosons. Based on joint works with S. Breteaux, J. Faupin, D.H. Ou Yang, I.M. Sigal, and J. Zhang.
Harvard Science Center, 1 Oxford Street, Cambridge MA 02138
Probability Seminar
Speaker: Arka Adhikari (Stanford)
Title: Correlation decay for finite lattice gauge theories
Abstract: In the setting of lattice gauge theories with finite (possibly non-Abelian) gauge groups at weak coupling, we prove exponential decay of correlations for a wide class of gauge invariant functions, which in particular includes arbitrary functions of Wilson loop observables. Based on joint work with Sky Cao.
Title: A 6-year journey: from gravitational anomaly to a unified theory of generalized symmetry
Abstract: Emergent symmetry can be generalized symmetry beyond (higher) group description and/or can be anomalous. I will describe a unified theory for generalized symmetry based on symmetry/topological-order correspondence. I will also discuss some applications of emergent generalized symmetry.
Title: Pole skipping, quasinormal modes, shockwaves and their connection to chaos
Abstract: A chaotic quantum system can be studied using the out-of-time-order correlator (OTOC). I will tell you about pole skipping — a recently discovered feature of the retarded Green’s function — that seems to also know things: things like the Lyapunov exponent and the butterfly velocity, which are important quantifiers of the OTOC. Then I will talk about a systematic way of deriving pole-skipping conditions for general holographic CFTs dual to classical bulk theories and how to use this framework to derive a few interesting statements including: (1) theories with higher spins generally violate the chaos bound; (2) the butterfly velocity calculated using pole skipping agrees with that calculated using shockwaves for arbitrary higher-derivative gravity coupled to ordinary matter; (3) shockwaves are related to a special type of quasinormal modes. As we will see, the techniques are entirely classically gravitational, which I will go through with a certain level of details.
Title: Homotopy classes of loops of Clifford unitaries
Abstract: We study Clifford locality-preserving unitaries and stabilizer Hamiltonians by means of Hermitian K-theory. We demonstrate how the notion of algebraic homotopy of modules over Laurent polynomial rings translates into the connectedness of two short-range entangled stabilizer Hamiltonians by a shallow Clifford circuit. We apply this observation to a classification of homotopy classes of loops of Clifford unitaries. The talk is based on a work in collaboration with Yichen Hu. https://arxiv.org/abs/2306.09903.
Title: Phase transitions out of quantum Hall states in moire TMD bilayers
Abstract: Motivated by the recent experimental breakthroughs in observing Fractional Quantum Anomalous Hall (FQAH) states in moir\’e Transition Metal Dichalcogenide (TMD) bilayers, we propose and study various unconventional phase transitions between quantum Hall phases and Fermi liquids or charge ordered phases upon tuning the bandwidth. At filling -2/3, we describe a direct transition between the FQAH state and a Charge Density Wave (CDW) insulator. The critical theory resembles that of the familiar deconfined quantum critical point (DQCP) but with an additional Chern-Simons term. At filling -1/2, we study the possibility of a continuous transition between the composite Fermi liquid (CFL) and the Fermi liquid (FL) building on and refining previous work by Barkeshli and McGreevy. Crucially we show that translation symmetry alone is enough to enable a second order CFL-FL transition. We argue that there must be critical CDW fluctuations though neither phase has long range CDW order. A striking signature is a universal jump of resistivities at the critical point. With disorder, we argue that the CDW order gets pinned and the CFL-FL evolution happens through an intermediate electrically insulating phase with mobile neutral fermions. A clean analog of this insulating phase with long range CDW order and a neutral fermi surface can potentially also exist. We also present a critical theory for the CFL to FL transition at filling -3/4. Our work opens up a new avenue to realize deconfined criticality and fractionalized phases beyond familiar Landau level physics in the moire Chern band system.
Title: Frustration-free states of cell fate networks: the case of the epithelial-mesenchymal transition
Abstract: Cell fate decisions are made by allowing external signals to govern the steady-state pattern adopted by networks of interacting regulatory factors governing transcription and translation. One of these decisions, of importance for both developmental processes and for cancer metastasis, is the epithelial-mesenchymal transition (EMT). In this talk, we will argue that these biological networks have highly non-generic interaction structures such that they allow for phenotypic states with very low frustration, i.e. where most interactions are satisfied. This property has important consequences for the allowed dynamics of these systems.
Title: Quantum UV-IR map and curve counts in skeins
Abstract: Quantum UV-IR map (a.k.a. q-nonabelianization map), introduced by Neitzke and Yan, is a map from UV line defects in a 4d N=2 theory of class S to those of the IR. Mathematically, it can be described as a map between skein modules and is a close cousin of quantum trace map of Bonahon and Wong.
In this talk, I will discuss how quantum UV-IR map can be generalized to a map between HOMFLYPT skein modules, using skein-valued curve counts of Ekholm and Shende.
Title: Quantization of causal diamonds in 2+1 dimensional gravity
Abstract: We develop the reduced phase space quantization of causal diamonds in $2+1$ dimensional gravity with a nonpositive cosmological constant. The system is defined as the domain of dependence of a spacelike topological disk with a fixed boundary metric. By solving the constraints in a constant-mean-curvature time gauge and removing all the spatial gauge redundancy, we find that the phase space is the cotangent bundle of $Diff^+(S^1)/PSL(2, \mathbb{R})$, i.e., the group of orientation-preserving diffeomorphisms of the circle modulo the projective special linear subgroup. Classically, the states correspond to causal diamonds embedded in $AdS_3$ (or $Mink_3$ if $\Lambda = 0$), with a fixed corner length, that has the topological disk as a Cauchy surface. Because this phase space does not admit a global system of coordinates, a generalization of the standard canonical (coordinate) quantization is required — in particular, since the configuration space is a homogeneous space for a Lie group, we apply Isham’s group-theoretic quantization scheme. The Hilbert space of the associated quantum theory carries an irreducible unitary representation of the $BMS_3$ group and can be realized by wavefunctions on a coadjoint orbit of Virasoro with labels in irreducible unitary representations of the corresponding little group. A surprising result is that the twist of the diamond boundary loop is quantized in terms of the ratio of the Planck length to the corner length.
Speaker: Jonah Herzog-Arbeitman, Princeton University
Title: Exact Results in Flat Band Hubbard Models
Abstract: Flat bands, like those in the kagome lattice or twisted bilayer graphene, are a natural setting for studying strongly coupled physics since the interaction strength is the only energy scale in the problem. They can exhibit unconventional behavior in the multi-orbital case: the mean-field theory of flat band attractive Hubbard models shows the possibility of superconductivity even though the Fermi velocity of the bands is strictly zero. However, it is not necessary to resort to this approximation. We demonstrate that the groundstates and low-energy excitations of a large class of attractive Hubbard models are exactly solvable, offering a rare, microscopic view of their physics. The solution reveals the importance of quantum geometry in escaping (some of) BCS phenomenology within a tractable and nontrivial strong coupling theory.
Abstract: Scattering amplitudes in strong background fields provide an arena where perturbative and non-perturbative physics meet, with important applications ranging from laser physics to black holes, but their study is hampered by the cumbersome nature of QFT in the background field formalism. In this talk, I will try to convince you that strong-field scattering amplitudes contain a wealth of physical information which cannot be obtained with standard perturbative techniques, ranging from all-order classical observables to constraints on exact solutions. Furthermore, I will discuss how amplitudes in certain chiral strong fields can be obtained to all-multiplicity twistor and string methods.
Title: The TinyStories Dataset: How Small Can Language Models Be And Still Speak Coherent
Abstract: While generative language models exhibit powerful capabilities at large scale, when either the model or the number of training steps is too small, they struggle to produce coherent and fluent text: Existing models whose size is below a few billion parameters often do not generate coherent text beyond a few sentences. Hypothesizing that one of the main reasons for the strong reliance on size is the vast breadth and abundance of patterns in the datasets used to train those models, this motivates the following question: Can we design a dataset that preserves the essential elements of natural language, such as grammar, vocabulary, facts, and reasoning, but that is much smaller and more refined in terms of its breadth and diversity?
In this talk, we introduce TinyStories, a synthetic dataset of short stories that only contain words that 3 to 4-year-olds typically understand, generated by GPT-3.5/4. We show that TinyStories can be used to train and analyze language models that are much smaller than the state-of-the-art models (below 10 million parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate certain reasoning capabilities. We also show that the trained models are substantially more interpretable than larger ones, as we can visualize and analyze the attention and activation patterns of the models, and show how they relate to the generation process and the story content. We hope that TinyStories can facilitate the development, analysis and research of language models, especially for low-resource or specialized domains, and shed light on the emergence of language capabilities in LMs.
Speaker: Nicola Kistler (Johann Wolfgang Goethe-Universität Frankfurt am Main)
Title: Solving spin systems, the Babylonian way
Abstract: The replica method, together with Parisi’s symmetry breaking mechanism, is an extremely powerful tool to compute the limiting free energy of virtually any mean field disordered system. Unfortunately, the tool is dramatically flawed from a mathematical point of view. I will discuss a truly elementary procedure which allows to rigorously implement two (out of three) steps of the replica method, and conclude with some remarks on the relation between this new point of view and old work by Mezard and Virasoro on the microstructure of ultrametricity, the latter being the fundamental yet unjustified Ansatz in the celebrated Parisi solution. We are still far from a clear understanding of the issues, but quite astonishingly, evidence is mounting that Parisi’s ultrametricity assumption, the onset of scales and the universal hierarchical self-organisation of random systems in the infinite volume limit, is intimately linked to hidden geometrical properties of large random matrices which satisfy rules reminiscent of the popular SUDOKU game.
Speaker: Jonah Herzog-Arbeitman, Princeton University
Title: Exact Results in Flat Band Hubbard Models
Abstract: Flat bands, like those in the kagome lattice or twisted bilayer graphene, are a natural setting for studying strongly coupled physics since the interaction strength is the only energy scale in the problem. They can exhibit unconventional behavior in the multi-orbital case: the mean-field theory of flat band attractive Hubbard models shows the possibility of superconductivity even though the Fermi velocity of the bands is strictly zero. However, it is not necessary to resort to this approximation. We demonstrate that the groundstates and low-energy excitations of a large class of attractive Hubbard models are exactly solvable, offering a rare, microscopic view of their physics. The solution reveals the importance of quantum geometry in escaping (some of) BCS phenomenology within a tractable and nontrivial strong coupling theory.
Abstract: Scattering amplitudes in strong background fields provide an arena where perturbative and non-perturbative physics meet, with important applications ranging from laser physics to black holes, but their study is hampered by the cumbersome nature of QFT in the background field formalism. In this talk, I will try to convince you that strong-field scattering amplitudes contain a wealth of physical information which cannot be obtained with standard perturbative techniques, ranging from all-order classical observables to constraints on exact solutions. Furthermore, I will discuss how amplitudes in certain chiral strong fields can be obtained to all-multiplicity twistor and string methods.
Title: The TinyStories Dataset: How Small Can Language Models Be And Still Speak Coherent
Abstract: While generative language models exhibit powerful capabilities at large scale, when either the model or the number of training steps is too small, they struggle to produce coherent and fluent text: Existing models whose size is below a few billion parameters often do not generate coherent text beyond a few sentences. Hypothesizing that one of the main reasons for the strong reliance on size is the vast breadth and abundance of patterns in the datasets used to train those models, this motivates the following question: Can we design a dataset that preserves the essential elements of natural language, such as grammar, vocabulary, facts, and reasoning, but that is much smaller and more refined in terms of its breadth and diversity?
In this talk, we introduce TinyStories, a synthetic dataset of short stories that only contain words that 3 to 4-year-olds typically understand, generated by GPT-3.5/4. We show that TinyStories can be used to train and analyze language models that are much smaller than the state-of-the-art models (below 10 million parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate certain reasoning capabilities. We also show that the trained models are substantially more interpretable than larger ones, as we can visualize and analyze the attention and activation patterns of the models, and show how they relate to the generation process and the story content. We hope that TinyStories can facilitate the development, analysis and research of language models, especially for low-resource or specialized domains, and shed light on the emergence of language capabilities in LMs.
Speaker: Nicola Kistler (Johann Wolfgang Goethe-Universität Frankfurt am Main)
Title: Solving spin systems, the Babylonian way
Abstract: The replica method, together with Parisi’s symmetry breaking mechanism, is an extremely powerful tool to compute the limiting free energy of virtually any mean field disordered system. Unfortunately, the tool is dramatically flawed from a mathematical point of view. I will discuss a truly elementary procedure which allows to rigorously implement two (out of three) steps of the replica method, and conclude with some remarks on the relation between this new point of view and old work by Mezard and Virasoro on the microstructure of ultrametricity, the latter being the fundamental yet unjustified Ansatz in the celebrated Parisi solution. We are still far from a clear understanding of the issues, but quite astonishingly, evidence is mounting that Parisi’s ultrametricity assumption, the onset of scales and the universal hierarchical self-organisation of random systems in the infinite volume limit, is intimately linked to hidden geometrical properties of large random matrices which satisfy rules reminiscent of the popular SUDOKU game.
Title: Floquet codes, automorphisms, and quantum computation
Abstract: In this talk, I will introduce a new kind of measurement-based quantum computation inspired by Floquet codes. In this model, the quantum logical gates are implemented by short sequences of low-weight measurements which simultaneously encode logical information and enable error correction. We introduce a new class of quantum error-correcting codes generalizing Floquet codes that achieve this, which we call dynamic automorphism (DA) codes.
As in Floquet codes, the instantaneous codespace of a DA code at any fixed point in time is that of a topological code. In this case, the quantum computation can be viewed as a sequence of time-like domain walls implementing automorphisms of the topological order, which can be understood in terms of reversible anyon condensation paths in a particular parent model. This talk will introduce all of these concepts as well as provide a new perspective for thinking about Floquet codes.
The explicit examples that we construct, which we call DA color codes, can implement the full Clifford group of logical gates in 2+1d by two- and, rarely three-body measurements. Using adaptive two-body measurements, we can achieve a non-Clifford gate in 3+1d, making the first step towards universal quantum computation in this model.
The talk is based on recent work with Nathanan Tantivasadakarn, Shankar Balasubramanian, and David Aasen [arxiv: 2307.10353].
Abstract: Fano varieties are the basic building blocks of algebraic varieties. Smooth Fano varieties have been classified in dimensions one (the projective line), two (del Pezzo surfaces), and three (Mori-Mukai classification). What does Mirror Symmetry have to say about such classifications? By studying the Landau-Ginzburg models mirror to smooth Fano threefolds we can transform the Mori-Mukai classification into an effective uniruledness result for moduli spaces of certain K3 and abelian surfaces. This is joint work with Andrew Harder, Ludmil Katzarkov, Mikhail Ovcharenko, and Victor Przjalkowski (arXiv:2307.15607).
Pre-talk Speaker: David Wu (Harvard Physics): 10:00-10:30 am
Speaker: Damian van de Heisteeg, CMSA
Title: Species Scale across String Moduli Spaces
Abstract: String theories feature a wide array of moduli spaces. We propose that the energy cutoff scale of these theories – the so-called species scale – can be determined through higher-curvature corrections. This species scale varies with the moduli; we use it both asymptotically to bound the diameter of the field space, as well as in the interior to determine a “desert point” where it is maximized.
Speaker: Sean Cox, Virginia Commonwealth University
Title: Predicting non-continuous functions
Abstract: One of the strangest consequences of the Axiom of Choice is the following Hardin-Taylor 2008 result: there is a “predictor” such that for every function $f$ from the reals to the reals—even nowhere continuous $f$—the predictor applied to $f \restriction (-\infty,t)$ correctly predicts $f(t)$ for *almost every* $t \in R$. They asked how robust such a predictor could be, with respect to distortions in the time (input) axis; more precisely, for which subgroups $H$ of Homeo^+(R) do there exist $H$-invariant predictors? Bajpai-Velleman proved an affirmative answer when H=Affine^+(R), and a negative answer when H is (the subgroup generated by) C^\infty(R). They asked about the intermediate region; in particular, do there exist analytic-invariant predictors? We have partially answered that question: assuming the Continuum Hypothesis (CH), the answer is “no”. Regarding other subgroups of Homeo^+(R), we have affirmative answers that rely solely on topological group-theoretic properties of the subgroup. But these properties are very restrictive; e.g., all known positive examples are metabelian. So there remain many open questions. This is joint work with Aldi, Buffkin, Cline, Cody, Elpers, and Lee.
Title: Geometry at Strong coupling for amplitudes/Wilson loops
Abstract: The amplitude/Wilson loop correspondence identifies planar N=4 super-Yang-Mills amplitudes with certain null polygonal Wilson loops at all the values of the coupling. At strong coupling this equates the amplitude/Wilson loop computed by Alday & Maldacena in terms of the area of a minimal surface in AdS_5. To do so they developed a `Y-system’ for computing the amplitude. This talk re-interprets their construction as providing the underlying twistor space for a hyperKahler structure on the corresponding space of kinematic data. In particular, the area is given by a Kahler scalar for the pseudo-hyperkahler structure and satisfies a version of the Plebanski equations, a well-known completely integrable system. This geometry encodes the properties of the space of kinematic data on which the amplitude depends as a cluster variety tying into its positive geometry and cluster variety structure. Similar constructures are possible for other cluster varieties corresponding to form factors and beyond.
Title: Transformers for maths, and maths for transformers
Abstract: Transformers can be trained to solve problems of mathematics. I present two recent applications, in mathematics and physics: predicting integer sequences, and discovering the properties of scattering amplitudes in a close relative of Quantum ChromoDynamics.
Problems of mathematics can also help understand transformers. Using two examples from linear algebra and integer arithmetic, I show that model predictions can be explained, that trained models do not confabulate, and that carefully choosing the training distributions can help achieve better, and more robust, performance.
Abstract: A dimer tiling of Z^d is a collection of edges such that every vertex is covered exactly once. In 2000, Cohn, Kenyon, and Propp showed that 2D dimer tilings satisfy a large deviations principle. In joint work with Nishant Chandgotia and Scott Sheffield, we prove an analogous large deviations principle for dimers in 3D. A lot of the results for dimers in two dimensions use tools and exact formulas (e.g. the height function representation of a tiling or the Kasteleyn determinant formula) that are specific to dimension 2. In this talk, I will try to give some intuition for why three dimensions is different from two, explain how to formulate the large deviations principle in 3D, show simulations, and explain some of the ways that we use a smaller set of tools (e.g. Hall’s matching theorem or a double dimer swapping operation) in our arguments.
Speaker: Max Lavrentovich, Worcester State University
Title: Strongly driven mixtures and membranes: Out of equilibrium surprises
Abstract: The more prosaic cousin of active matter, driven inactive matter, is still full of unexpected phenomena. I will discuss two projects involving two seemingly mundane systems, a phase-separating colloidal mixture and a lipid membrane, which demonstrate counterintuitive properties when driven out of equilibrium. We will see that the phase separating mixture, when driven by a uniform force, develops (in simulations) an intriguing pattern with a characteristic length scale set by the magnitude of the drive. We will look at some theoretical approaches to understanding the pattern formation and possible experimental realizations. The membrane, when driven by an oscillatory electric field, develops (in experiments) a long-lived metastable state with a decreased capacitance and increased dissipation. This state may have implications for neuronal processing and memory formation.
Title: Quantum field theory approach to quantum information
Abstract: We apply the formalism of quantum field theory and Euclidean space-time path integral to investigate a class of quantum information problems. In particular, we investigate quantum many-body systems under weak-measurement and decoherence. The Euclidean space-time path integral allows us to map this problem to a quantum field theory with (temporal) boundary or defects. We therefore investigate two types of quantum many-body systems with nontrivial boundary physics: quantum critical points, and states with nontrivial topology, such as Chern insulator and symmetry protected topological states. For example, we demonstrate that a Wilson-Fisher quantum critical point can be driven into an “extraordinary-log” phase after weak-measurement. Another example is that, we argue that a system with higher form symmetry may be driven to a self-dual phase transition under weak measurement.
Title: Moduli of vector bundles on curve and semiorthogonal decomposition
Abstract: In this talk we construct semiorthogonal decompositions of moduli of vector bundles on a curve into its symmetric powers. The essential ingredients in the proof include Borel-Weil-Bott theory for loop groups, derived Schur-Weyl duality for current groups and derived Θ-stratification.
Title: Topological modular forms and heretoric string theory
Abstract: In this talk I will explain my works with Y. Tachikawa to study anomaly in heterotic string theory via homotopy theory, especially the theory of Topological Modular Forms (TMF). TMF is an E-infinity ring spectrum which is conjectured by Stolz-Teichner to classify two-dimensional supersymmetric quantum field theories in physics. In the previous work (https://arxiv.org/abs/2108.13542), we proved the vanishing of anomalies in heterotic string theory mathematically by using TMF.
Furthermore, we have a recent update (https://arxiv.org/abs/2305.06196) on the previous work. Because of the vanishing result, we can consider a secondary transformation of spectra, which is shown to coincide with the Anderson self-duality morphism of TMF. This allows us to detect subtle torsion phenomena in TMF by differential-geometric ways, and leads us to new conjectures on the relation between VOAs and TMF.
On August 31-Sep 1, 2023 the CMSA will host the ninth annual Conference on Big Data. The Big Data Conference features speakers from the Harvard community as well as scholars from across the globe, with talks focusing on computer science, statistics, math and physics, and economics.
Location: Harvard University Science Center Hall D & via Zoom.
Title: From Network Medicine to the Foodome: The Dark Matter of Nutrition
Abstract: A disease is rarely a consequence of an abnormality in a single gene but reflects perturbations to the complex intracellular network. Network medicine offer a platform to explore systematically not only the molecular complexity of a particular disease, leading to the identification of disease modules and pathways, but also the molecular relationships between apparently distinct (patho) phenotypes. As an application, I will explore how we use network medicine to uncover the role individual food molecules in our health. Indeed, our current understanding of how diet affects our health is limited to the role of 150 key nutritional components systematically tracked by the USDA and other national databases in all foods. Yet, these nutritional components represent only a tiny fraction of the over 135,000 distinct, definable biochemicals present in our food. While many of these biochemicals have documented effects on health, they remain unquantified in any systematic fashion across different individual foods. Their invisibility to experimental, clinical, and epidemiological studies defines them as the ‘Dark Matter of Nutrition.’ I will speak about our efforts to develop a high-resolution library of this nutritional dark matter, and efforts to understand the role of these molecules on health, opening novel avenues by which to understand, avoid, and control disease.
Title: Differentially Private Algorithms for Statistical Estimation Problems
Abstract: Differential privacy (DP) is widely regarded as a gold standard for privacy-preserving computation over users’ data. It is a parameterized notion of database privacy that gives a rigorous worst-case bound on the information that can be learned about any one individual from the result of a data analysis task. Algorithmically it is achieved by injecting carefully calibrated randomness into the analysis to balance privacy protections with accuracy of the results. In this talk, we will survey recent developments in the development of DP algorithms for three important statistical problems, namely online learning with bandit feedback, causal interference, and learning from imbalanced data. For the first problem, we will show that Thompson sampling — a standard bandit algorithm developed in the 1930s — already satisfies DP due to the inherent randomness of the algorithm. For the second problem of causal inference and counterfactual estimation, we develop the first DP algorithms for synthetic control, which has been used non-privately for this task for decades. Finally, for the problem of imbalanced learning, where one class is severely underrepresented in the training data, we show that combining existing techniques such as minority oversampling perform very poorly when applied as pre-processing before a DP learning algorithm; instead we propose novel approaches for privately generating synthetic minority points.
Based on joint works with Marco Avella Medina, Vishal Misra, Yuliia Lut, Tingting Ou, Saeyoung Rho, and Ethan Turok.
Title: To split or not to split that is the question: From cross validation to debiased machine learning
Abstract: Data splitting is a ubiquitous method in statistics with examples ranging from cross-validation to cross-fitting. However, despite its prevalence, theoretical guidance regarding its use is still lacking. In this talk, we will explore two examples and establish an asymptotic theory for it. In the first part of this talk, we study the cross-validation method, a ubiquitous method for risk estimation, and establish its asymptotic properties for a large class of models and with an arbitrary number of folds. Under stability conditions, we establish a central limit theorem and Berry-Esseen bounds for the cross-validated risk, which enable us to compute asymptotically accurate confidence intervals. Using our results, we study the statistical speed-up offered by cross-validation compared to a train-test split procedure. We reveal some surprising behavior of the cross-validated risk and establish the statistically optimal choice for the number of folds. In the second part of this talk, we study the role of cross-fitting in the generalized method of moments with moments that also depend on some auxiliary functions. Recent lines of work show how one can use generic machine learning estimators for these auxiliary problems, while maintaining asymptotic normality and root-n consistency of the target parameter of interest. The literature typically requires that these auxiliary problems are fitted on a separate sample or in a cross-fitting manner. We show that when these auxiliary estimation algorithms satisfy natural leave-one-out stability properties, then sample splitting is not required. This allows for sample reuse, which can be beneficial in moderately sized sample regimes.
Abstract: Linear dynamical systems are the canonical model for time series data. They have wide-ranging applications and there is a vast literature on learning their parameters from input-output sequences. Moreover they have received renewed interest because of their connections to recurrent neural networks. But there are wide gaps in our understanding. Existing works have only asymptotic guarantees or else make restrictive assumptions, e.g. that preclude having any long-range correlations. In this work, we give a new algorithm based on the method of moments that is computationally efficient and works under essentially minimal assumptions. Our work points to several missed connections, whereby tools from theoretical machine learning including tensor methods, can be used in non-stationary settings.
Title: Algorithmic Thresholds for Spherical Spin Glasses
Abstract: High-dimensional optimization plays a crucial role in modern statistics and machine learning. I will present recent progress on non-convex optimization problems with random objectives, focusing on the spherical p-spin glass. This model is related to spiked tensor estimation and has been studied in probability and physics for decades. We will see that a natural class of “stable” optimization algorithms gets stuck at an algorithmic threshold related to geometric properties of the landscape. The algorithmic threshold value is efficiently attained via Langevin dynamics or by a second-order ascent method of Subag. Much of this picture extends to other models, such as random constraint satisfaction problems at high clause density.
Title: What Learning Algorithm is In-Context Learning?
Abstract: Neural sequence models, especially transformers, exhibit a remarkable capacity for “in-context” learning. They can construct new predictors from sequences of labeled examples (x,f(x)) presented in the input without further parameter updates. I’ll present recent findings suggesting that transformer-based in-context learners implement standard learning algorithms implicitly, by encoding smaller models in their activations, and updating these implicit models as new examples appear in the context, using in-context linear regression as a model problem. First, I’ll show by construction that transformers can implement learning algorithms for linear models based on gradient descent and closed-form ridge regression. Second, I’ll show that trained in-context learners closely match the predictors computed by gradient descent, ridge regression, and exact least-squares regression, transitioning between different predictors as transformer depth and dataset noise vary, and converging to Bayesian estimators for large widths and depths. Finally, we present preliminary evidence that in-context learners share algorithmic features with these predictors: learners’ late layers non-linearly encode weight vectors and moment matrices. These results suggest that in-context learning is understandable in algorithmic terms, and that (at least in the linear case) learners may rediscover standard estimation algorithms. This work is joint with Ekin Akyürek at MIT, and Dale Schuurmans, Tengyu Ma and Denny Zhou at Stanford.
Abstract: Rapidly advancing deep distributional modeling techniques offer a number of opportunities for complex generative tasks, from natural sciences such as molecules and materials to engineering. I will discuss generative approaches inspired from physical processes including diffusion models and more recent electrostatic models (Poisson flow), and how they relate to each other in terms of embedding dimension. From the point of view of applications, I will highlight our recent work on SE(3) invariant distributional modeling over backbone 3D structures with ability to generate designable monomers without relying on pre-trained protein structure prediction methods as well as state of the art image generation capabilities (Poisson flow). Time permitting, I will also discuss recent analysis of efficiency of sample generation in such models.
Abstract: Understanding biological and natural systems requires modeling data with underlying geometric relationships across scales and modalities such as biological sequences, chemical constraints, and graphs of 3D spatial or biological interactions. I will discuss unique challenges for learning from multimodal datasets that are due to varying inductive biases across modalities and the potential absence of explicit graphs in the input. I will describe a framework for structure-inducing pretraining that allows for a comprehensive study of how relational structure can be induced in pretrained language models. We use the framework to explore new graph pretraining objectives that impose relational structure in the induced latent spaces—i.e., pretraining objectives that explicitly impose structural constraints on the distance or geometry of pretrained models. Applications in genomic medicine and therapeutic science will be discussed. These include TxGNN, an AI model enabling zero-shot prediction of therapeutic use across over 17,000 diseases, and PINNACLE, a contextual graph AI model dynamically adjusting its outputs to contexts in which it operates. PINNACLE enhances 3D protein structure representations and predicts the effects of drugs at single-cell resolution.
Abstract: Uncertainty quantification for prediction is an intriguing problem with significant applications in various fields, such as biomedical science, economic studies, and weather forecasts. Numerous methods are available for constructing prediction intervals, such as quantile regression and conformal predictions, among others. Nevertheless, model misspecification (especially in high-dimension) or sub-optimal constructions can frequently result in biased or unnecessarily-wide prediction intervals. In this work, we propose a novel and widely applicable technique for aggregating multiple prediction intervals to minimize the average width of the prediction band along with coverage guarantee, called Universally Trainable Optimal Predictive Intervals Aggregation (UTOPIA). The method also allows us to directly construct predictive bands based on elementary basis functions. Our approach is based on linear or convex programming which is easy to implement. All of our proposed methodologies are supported by theoretical guarantees on the coverage probability and optimal average length, which are detailed in this paper. The effectiveness of our approach is convincingly demonstrated by applying it to synthetic data and two real datasets on finance and macroeconomics. (Joint work Jiawei Ge and Debarghya Mukherjee).
Title: Efficient OCR for Building a Diverse Digital History
Abstract: Many users consult digital archives daily, but the information they can access is unrepresentative of the diversity of documentary history. The sequence-to-sequence architecture typically used for optical character recognition (OCR) – which jointly learns a vision and language model – is poorly extensible to low-resource document collections, as learning a language-vision model requires extensive labeled sequences and compute. This study models OCR as a character-level image retrieval problem, using a contrastively trained vision encoder. Because the model only learns characters’ visual features, it is more sample-efficient and extensible than existing architectures, enabling accurate OCR in settings where existing solutions fail. Crucially, it opens new avenues for community engagement in making digital history more representative of documentary history.
Harvard Science Center, 1 Oxford Street, Cambridge MA 02138
Probability Seminar
Speaker: Marius Lemm, University of Tuebingen
Title: Light cones for open quantum systems
Abstract: We consider non-relativistic Markovian open quantum dynamics in continuous space. We show that, up to small probability tails, the supports of quantum states propagate with finite speed in any finite-energy subspace. More precisely, if the initial quantum state is localized in space, then any finite-energy part of the solution of the von Neumann-Lindblad equation is approximately localized inside an energy-dependent light cone. We also obtain an explicit upper bound on the slope of this light cone (i.e., on the maximal speed). The general method can be used to derive propagation bounds for a variety of other quantum systems including Lieb-Robinson bounds for lattice bosons. Based on joint works with S. Breteaux, J. Faupin, D.H. Ou Yang, I.M. Sigal, and J. Zhang.
Harvard Science Center, 1 Oxford Street, Cambridge MA 02138
Probability Seminar
Speaker: Arka Adhikari (Stanford)
Title: Correlation decay for finite lattice gauge theories
Abstract: In the setting of lattice gauge theories with finite (possibly non-Abelian) gauge groups at weak coupling, we prove exponential decay of correlations for a wide class of gauge invariant functions, which in particular includes arbitrary functions of Wilson loop observables. Based on joint work with Sky Cao.
Title: A 6-year journey: from gravitational anomaly to a unified theory of generalized symmetry
Abstract: Emergent symmetry can be generalized symmetry beyond (higher) group description and/or can be anomalous. I will describe a unified theory for generalized symmetry based on symmetry/topological-order correspondence. I will also discuss some applications of emergent generalized symmetry.
Title: Pole skipping, quasinormal modes, shockwaves and their connection to chaos
Abstract: A chaotic quantum system can be studied using the out-of-time-order correlator (OTOC). I will tell you about pole skipping — a recently discovered feature of the retarded Green’s function — that seems to also know things: things like the Lyapunov exponent and the butterfly velocity, which are important quantifiers of the OTOC. Then I will talk about a systematic way of deriving pole-skipping conditions for general holographic CFTs dual to classical bulk theories and how to use this framework to derive a few interesting statements including: (1) theories with higher spins generally violate the chaos bound; (2) the butterfly velocity calculated using pole skipping agrees with that calculated using shockwaves for arbitrary higher-derivative gravity coupled to ordinary matter; (3) shockwaves are related to a special type of quasinormal modes. As we will see, the techniques are entirely classically gravitational, which I will go through with a certain level of details.
Title: Homotopy classes of loops of Clifford unitaries
Abstract: We study Clifford locality-preserving unitaries and stabilizer Hamiltonians by means of Hermitian K-theory. We demonstrate how the notion of algebraic homotopy of modules over Laurent polynomial rings translates into the connectedness of two short-range entangled stabilizer Hamiltonians by a shallow Clifford circuit. We apply this observation to a classification of homotopy classes of loops of Clifford unitaries. The talk is based on a work in collaboration with Yichen Hu. https://arxiv.org/abs/2306.09903.
Title: Phase transitions out of quantum Hall states in moire TMD bilayers
Abstract: Motivated by the recent experimental breakthroughs in observing Fractional Quantum Anomalous Hall (FQAH) states in moir\’e Transition Metal Dichalcogenide (TMD) bilayers, we propose and study various unconventional phase transitions between quantum Hall phases and Fermi liquids or charge ordered phases upon tuning the bandwidth. At filling -2/3, we describe a direct transition between the FQAH state and a Charge Density Wave (CDW) insulator. The critical theory resembles that of the familiar deconfined quantum critical point (DQCP) but with an additional Chern-Simons term. At filling -1/2, we study the possibility of a continuous transition between the composite Fermi liquid (CFL) and the Fermi liquid (FL) building on and refining previous work by Barkeshli and McGreevy. Crucially we show that translation symmetry alone is enough to enable a second order CFL-FL transition. We argue that there must be critical CDW fluctuations though neither phase has long range CDW order. A striking signature is a universal jump of resistivities at the critical point. With disorder, we argue that the CDW order gets pinned and the CFL-FL evolution happens through an intermediate electrically insulating phase with mobile neutral fermions. A clean analog of this insulating phase with long range CDW order and a neutral fermi surface can potentially also exist. We also present a critical theory for the CFL to FL transition at filling -3/4. Our work opens up a new avenue to realize deconfined criticality and fractionalized phases beyond familiar Landau level physics in the moire Chern band system.
Title: Frustration-free states of cell fate networks: the case of the epithelial-mesenchymal transition
Abstract: Cell fate decisions are made by allowing external signals to govern the steady-state pattern adopted by networks of interacting regulatory factors governing transcription and translation. One of these decisions, of importance for both developmental processes and for cancer metastasis, is the epithelial-mesenchymal transition (EMT). In this talk, we will argue that these biological networks have highly non-generic interaction structures such that they allow for phenotypic states with very low frustration, i.e. where most interactions are satisfied. This property has important consequences for the allowed dynamics of these systems.
Title: Quantum UV-IR map and curve counts in skeins
Abstract: Quantum UV-IR map (a.k.a. q-nonabelianization map), introduced by Neitzke and Yan, is a map from UV line defects in a 4d N=2 theory of class S to those of the IR. Mathematically, it can be described as a map between skein modules and is a close cousin of quantum trace map of Bonahon and Wong.
In this talk, I will discuss how quantum UV-IR map can be generalized to a map between HOMFLYPT skein modules, using skein-valued curve counts of Ekholm and Shende.
Title: Quantization of causal diamonds in 2+1 dimensional gravity
Abstract: We develop the reduced phase space quantization of causal diamonds in $2+1$ dimensional gravity with a nonpositive cosmological constant. The system is defined as the domain of dependence of a spacelike topological disk with a fixed boundary metric. By solving the constraints in a constant-mean-curvature time gauge and removing all the spatial gauge redundancy, we find that the phase space is the cotangent bundle of $Diff^+(S^1)/PSL(2, \mathbb{R})$, i.e., the group of orientation-preserving diffeomorphisms of the circle modulo the projective special linear subgroup. Classically, the states correspond to causal diamonds embedded in $AdS_3$ (or $Mink_3$ if $\Lambda = 0$), with a fixed corner length, that has the topological disk as a Cauchy surface. Because this phase space does not admit a global system of coordinates, a generalization of the standard canonical (coordinate) quantization is required — in particular, since the configuration space is a homogeneous space for a Lie group, we apply Isham’s group-theoretic quantization scheme. The Hilbert space of the associated quantum theory carries an irreducible unitary representation of the $BMS_3$ group and can be realized by wavefunctions on a coadjoint orbit of Virasoro with labels in irreducible unitary representations of the corresponding little group. A surprising result is that the twist of the diamond boundary loop is quantized in terms of the ratio of the Planck length to the corner length.
Speaker: Jonah Herzog-Arbeitman, Princeton University
Title: Exact Results in Flat Band Hubbard Models
Abstract: Flat bands, like those in the kagome lattice or twisted bilayer graphene, are a natural setting for studying strongly coupled physics since the interaction strength is the only energy scale in the problem. They can exhibit unconventional behavior in the multi-orbital case: the mean-field theory of flat band attractive Hubbard models shows the possibility of superconductivity even though the Fermi velocity of the bands is strictly zero. However, it is not necessary to resort to this approximation. We demonstrate that the groundstates and low-energy excitations of a large class of attractive Hubbard models are exactly solvable, offering a rare, microscopic view of their physics. The solution reveals the importance of quantum geometry in escaping (some of) BCS phenomenology within a tractable and nontrivial strong coupling theory.
Abstract: Scattering amplitudes in strong background fields provide an arena where perturbative and non-perturbative physics meet, with important applications ranging from laser physics to black holes, but their study is hampered by the cumbersome nature of QFT in the background field formalism. In this talk, I will try to convince you that strong-field scattering amplitudes contain a wealth of physical information which cannot be obtained with standard perturbative techniques, ranging from all-order classical observables to constraints on exact solutions. Furthermore, I will discuss how amplitudes in certain chiral strong fields can be obtained to all-multiplicity twistor and string methods.
Title: The TinyStories Dataset: How Small Can Language Models Be And Still Speak Coherent
Abstract: While generative language models exhibit powerful capabilities at large scale, when either the model or the number of training steps is too small, they struggle to produce coherent and fluent text: Existing models whose size is below a few billion parameters often do not generate coherent text beyond a few sentences. Hypothesizing that one of the main reasons for the strong reliance on size is the vast breadth and abundance of patterns in the datasets used to train those models, this motivates the following question: Can we design a dataset that preserves the essential elements of natural language, such as grammar, vocabulary, facts, and reasoning, but that is much smaller and more refined in terms of its breadth and diversity?
In this talk, we introduce TinyStories, a synthetic dataset of short stories that only contain words that 3 to 4-year-olds typically understand, generated by GPT-3.5/4. We show that TinyStories can be used to train and analyze language models that are much smaller than the state-of-the-art models (below 10 million parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate certain reasoning capabilities. We also show that the trained models are substantially more interpretable than larger ones, as we can visualize and analyze the attention and activation patterns of the models, and show how they relate to the generation process and the story content. We hope that TinyStories can facilitate the development, analysis and research of language models, especially for low-resource or specialized domains, and shed light on the emergence of language capabilities in LMs.
Speaker: Nicola Kistler (Johann Wolfgang Goethe-Universität Frankfurt am Main)
Title: Solving spin systems, the Babylonian way
Abstract: The replica method, together with Parisi’s symmetry breaking mechanism, is an extremely powerful tool to compute the limiting free energy of virtually any mean field disordered system. Unfortunately, the tool is dramatically flawed from a mathematical point of view. I will discuss a truly elementary procedure which allows to rigorously implement two (out of three) steps of the replica method, and conclude with some remarks on the relation between this new point of view and old work by Mezard and Virasoro on the microstructure of ultrametricity, the latter being the fundamental yet unjustified Ansatz in the celebrated Parisi solution. We are still far from a clear understanding of the issues, but quite astonishingly, evidence is mounting that Parisi’s ultrametricity assumption, the onset of scales and the universal hierarchical self-organisation of random systems in the infinite volume limit, is intimately linked to hidden geometrical properties of large random matrices which satisfy rules reminiscent of the popular SUDOKU game.
Speaker: Jonah Herzog-Arbeitman, Princeton University
Title: Exact Results in Flat Band Hubbard Models
Abstract: Flat bands, like those in the kagome lattice or twisted bilayer graphene, are a natural setting for studying strongly coupled physics since the interaction strength is the only energy scale in the problem. They can exhibit unconventional behavior in the multi-orbital case: the mean-field theory of flat band attractive Hubbard models shows the possibility of superconductivity even though the Fermi velocity of the bands is strictly zero. However, it is not necessary to resort to this approximation. We demonstrate that the groundstates and low-energy excitations of a large class of attractive Hubbard models are exactly solvable, offering a rare, microscopic view of their physics. The solution reveals the importance of quantum geometry in escaping (some of) BCS phenomenology within a tractable and nontrivial strong coupling theory.
Abstract: Scattering amplitudes in strong background fields provide an arena where perturbative and non-perturbative physics meet, with important applications ranging from laser physics to black holes, but their study is hampered by the cumbersome nature of QFT in the background field formalism. In this talk, I will try to convince you that strong-field scattering amplitudes contain a wealth of physical information which cannot be obtained with standard perturbative techniques, ranging from all-order classical observables to constraints on exact solutions. Furthermore, I will discuss how amplitudes in certain chiral strong fields can be obtained to all-multiplicity twistor and string methods.
Title: The TinyStories Dataset: How Small Can Language Models Be And Still Speak Coherent
Abstract: While generative language models exhibit powerful capabilities at large scale, when either the model or the number of training steps is too small, they struggle to produce coherent and fluent text: Existing models whose size is below a few billion parameters often do not generate coherent text beyond a few sentences. Hypothesizing that one of the main reasons for the strong reliance on size is the vast breadth and abundance of patterns in the datasets used to train those models, this motivates the following question: Can we design a dataset that preserves the essential elements of natural language, such as grammar, vocabulary, facts, and reasoning, but that is much smaller and more refined in terms of its breadth and diversity?
In this talk, we introduce TinyStories, a synthetic dataset of short stories that only contain words that 3 to 4-year-olds typically understand, generated by GPT-3.5/4. We show that TinyStories can be used to train and analyze language models that are much smaller than the state-of-the-art models (below 10 million parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate certain reasoning capabilities. We also show that the trained models are substantially more interpretable than larger ones, as we can visualize and analyze the attention and activation patterns of the models, and show how they relate to the generation process and the story content. We hope that TinyStories can facilitate the development, analysis and research of language models, especially for low-resource or specialized domains, and shed light on the emergence of language capabilities in LMs.
Speaker: Nicola Kistler (Johann Wolfgang Goethe-Universität Frankfurt am Main)
Title: Solving spin systems, the Babylonian way
Abstract: The replica method, together with Parisi’s symmetry breaking mechanism, is an extremely powerful tool to compute the limiting free energy of virtually any mean field disordered system. Unfortunately, the tool is dramatically flawed from a mathematical point of view. I will discuss a truly elementary procedure which allows to rigorously implement two (out of three) steps of the replica method, and conclude with some remarks on the relation between this new point of view and old work by Mezard and Virasoro on the microstructure of ultrametricity, the latter being the fundamental yet unjustified Ansatz in the celebrated Parisi solution. We are still far from a clear understanding of the issues, but quite astonishingly, evidence is mounting that Parisi’s ultrametricity assumption, the onset of scales and the universal hierarchical self-organisation of random systems in the infinite volume limit, is intimately linked to hidden geometrical properties of large random matrices which satisfy rules reminiscent of the popular SUDOKU game.
Title: Floquet codes, automorphisms, and quantum computation
Abstract: In this talk, I will introduce a new kind of measurement-based quantum computation inspired by Floquet codes. In this model, the quantum logical gates are implemented by short sequences of low-weight measurements which simultaneously encode logical information and enable error correction. We introduce a new class of quantum error-correcting codes generalizing Floquet codes that achieve this, which we call dynamic automorphism (DA) codes.
As in Floquet codes, the instantaneous codespace of a DA code at any fixed point in time is that of a topological code. In this case, the quantum computation can be viewed as a sequence of time-like domain walls implementing automorphisms of the topological order, which can be understood in terms of reversible anyon condensation paths in a particular parent model. This talk will introduce all of these concepts as well as provide a new perspective for thinking about Floquet codes.
The explicit examples that we construct, which we call DA color codes, can implement the full Clifford group of logical gates in 2+1d by two- and, rarely three-body measurements. Using adaptive two-body measurements, we can achieve a non-Clifford gate in 3+1d, making the first step towards universal quantum computation in this model.
The talk is based on recent work with Nathanan Tantivasadakarn, Shankar Balasubramanian, and David Aasen [arxiv: 2307.10353].
Abstract: Fano varieties are the basic building blocks of algebraic varieties. Smooth Fano varieties have been classified in dimensions one (the projective line), two (del Pezzo surfaces), and three (Mori-Mukai classification). What does Mirror Symmetry have to say about such classifications? By studying the Landau-Ginzburg models mirror to smooth Fano threefolds we can transform the Mori-Mukai classification into an effective uniruledness result for moduli spaces of certain K3 and abelian surfaces. This is joint work with Andrew Harder, Ludmil Katzarkov, Mikhail Ovcharenko, and Victor Przjalkowski (arXiv:2307.15607).
Pre-talk Speaker: David Wu (Harvard Physics): 10:00-10:30 am
Speaker: Damian van de Heisteeg, CMSA
Title: Species Scale across String Moduli Spaces
Abstract: String theories feature a wide array of moduli spaces. We propose that the energy cutoff scale of these theories – the so-called species scale – can be determined through higher-curvature corrections. This species scale varies with the moduli; we use it both asymptotically to bound the diameter of the field space, as well as in the interior to determine a “desert point” where it is maximized.
Speaker: Sean Cox, Virginia Commonwealth University
Title: Predicting non-continuous functions
Abstract: One of the strangest consequences of the Axiom of Choice is the following Hardin-Taylor 2008 result: there is a “predictor” such that for every function $f$ from the reals to the reals—even nowhere continuous $f$—the predictor applied to $f \restriction (-\infty,t)$ correctly predicts $f(t)$ for *almost every* $t \in R$. They asked how robust such a predictor could be, with respect to distortions in the time (input) axis; more precisely, for which subgroups $H$ of Homeo^+(R) do there exist $H$-invariant predictors? Bajpai-Velleman proved an affirmative answer when H=Affine^+(R), and a negative answer when H is (the subgroup generated by) C^\infty(R). They asked about the intermediate region; in particular, do there exist analytic-invariant predictors? We have partially answered that question: assuming the Continuum Hypothesis (CH), the answer is “no”. Regarding other subgroups of Homeo^+(R), we have affirmative answers that rely solely on topological group-theoretic properties of the subgroup. But these properties are very restrictive; e.g., all known positive examples are metabelian. So there remain many open questions. This is joint work with Aldi, Buffkin, Cline, Cody, Elpers, and Lee.
Title: Geometry at Strong coupling for amplitudes/Wilson loops
Abstract: The amplitude/Wilson loop correspondence identifies planar N=4 super-Yang-Mills amplitudes with certain null polygonal Wilson loops at all the values of the coupling. At strong coupling this equates the amplitude/Wilson loop computed by Alday & Maldacena in terms of the area of a minimal surface in AdS_5. To do so they developed a `Y-system’ for computing the amplitude. This talk re-interprets their construction as providing the underlying twistor space for a hyperKahler structure on the corresponding space of kinematic data. In particular, the area is given by a Kahler scalar for the pseudo-hyperkahler structure and satisfies a version of the Plebanski equations, a well-known completely integrable system. This geometry encodes the properties of the space of kinematic data on which the amplitude depends as a cluster variety tying into its positive geometry and cluster variety structure. Similar constructures are possible for other cluster varieties corresponding to form factors and beyond.
Title: Transformers for maths, and maths for transformers
Abstract: Transformers can be trained to solve problems of mathematics. I present two recent applications, in mathematics and physics: predicting integer sequences, and discovering the properties of scattering amplitudes in a close relative of Quantum ChromoDynamics.
Problems of mathematics can also help understand transformers. Using two examples from linear algebra and integer arithmetic, I show that model predictions can be explained, that trained models do not confabulate, and that carefully choosing the training distributions can help achieve better, and more robust, performance.
Abstract: A dimer tiling of Z^d is a collection of edges such that every vertex is covered exactly once. In 2000, Cohn, Kenyon, and Propp showed that 2D dimer tilings satisfy a large deviations principle. In joint work with Nishant Chandgotia and Scott Sheffield, we prove an analogous large deviations principle for dimers in 3D. A lot of the results for dimers in two dimensions use tools and exact formulas (e.g. the height function representation of a tiling or the Kasteleyn determinant formula) that are specific to dimension 2. In this talk, I will try to give some intuition for why three dimensions is different from two, explain how to formulate the large deviations principle in 3D, show simulations, and explain some of the ways that we use a smaller set of tools (e.g. Hall’s matching theorem or a double dimer swapping operation) in our arguments.
Speaker: Max Lavrentovich, Worcester State University
Title: Strongly driven mixtures and membranes: Out of equilibrium surprises
Abstract: The more prosaic cousin of active matter, driven inactive matter, is still full of unexpected phenomena. I will discuss two projects involving two seemingly mundane systems, a phase-separating colloidal mixture and a lipid membrane, which demonstrate counterintuitive properties when driven out of equilibrium. We will see that the phase separating mixture, when driven by a uniform force, develops (in simulations) an intriguing pattern with a characteristic length scale set by the magnitude of the drive. We will look at some theoretical approaches to understanding the pattern formation and possible experimental realizations. The membrane, when driven by an oscillatory electric field, develops (in experiments) a long-lived metastable state with a decreased capacitance and increased dissipation. This state may have implications for neuronal processing and memory formation.
Title: Quantum field theory approach to quantum information
Abstract: We apply the formalism of quantum field theory and Euclidean space-time path integral to investigate a class of quantum information problems. In particular, we investigate quantum many-body systems under weak-measurement and decoherence. The Euclidean space-time path integral allows us to map this problem to a quantum field theory with (temporal) boundary or defects. We therefore investigate two types of quantum many-body systems with nontrivial boundary physics: quantum critical points, and states with nontrivial topology, such as Chern insulator and symmetry protected topological states. For example, we demonstrate that a Wilson-Fisher quantum critical point can be driven into an “extraordinary-log” phase after weak-measurement. Another example is that, we argue that a system with higher form symmetry may be driven to a self-dual phase transition under weak measurement.
Title: Moduli of vector bundles on curve and semiorthogonal decomposition
Abstract: In this talk we construct semiorthogonal decompositions of moduli of vector bundles on a curve into its symmetric powers. The essential ingredients in the proof include Borel-Weil-Bott theory for loop groups, derived Schur-Weyl duality for current groups and derived Θ-stratification.
Title: Topological modular forms and heretoric string theory
Abstract: In this talk I will explain my works with Y. Tachikawa to study anomaly in heterotic string theory via homotopy theory, especially the theory of Topological Modular Forms (TMF). TMF is an E-infinity ring spectrum which is conjectured by Stolz-Teichner to classify two-dimensional supersymmetric quantum field theories in physics. In the previous work (https://arxiv.org/abs/2108.13542), we proved the vanishing of anomalies in heterotic string theory mathematically by using TMF.
Furthermore, we have a recent update (https://arxiv.org/abs/2305.06196) on the previous work. Because of the vanishing result, we can consider a secondary transformation of spectra, which is shown to coincide with the Anderson self-duality morphism of TMF. This allows us to detect subtle torsion phenomena in TMF by differential-geometric ways, and leads us to new conjectures on the relation between VOAs and TMF.
