BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CMSA - ECPv6.15.20//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://cmsa.fas.harvard.edu
X-WR-CALDESC:Events for CMSA
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20240310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20241103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20260308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20261101T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20251017T120000
DTEND;TZID=America/New_York:20251017T130000
DTSTAMP:20260529T124307
CREATED:20250827T141359Z
LAST-MODIFIED:20251010T180544Z
UID:10003766-1760702400-1760706000@cmsa.fas.harvard.edu
SUMMARY:DMFT\, Two Point Correlations of Resolvents\, and Applications to Machine Learning Theory
DESCRIPTION:Member Seminar \nSpeaker: Blake Bordelon \nTitle: DMFT\, Two Point Correlations of Resolvents\, and Applications to Machine Learning Theory \nAbstract: Machine learning algorithms evolve the parameters of a model in a high dimensional and disordered loss landscape. To characterize the effects of random initialization of model parameters\, randomly sampled training data\, and the effect of SGD noise\, it often is useful to invoke ideas from random matrix theory and the physics of disordered systems. In this seminar\, I describe a general idea\, known as dynamical mean field theory (DMFT) which describes the evolution of a disordered dynamical system in infinite dimensions. I will briefly describe simple examples of interest to theoretical neuroscientists and machine learning theorists. For linear dynamical systems\, I will show that this method characterizes the typical case trajectory in terms of two point correlations of resolvent matrices evaluated at different frequencies. This bispectral object can account for puzzling effects such as late time divergence of gradient descent at the interpolation threshold (when parameters = dataset size) despite the Jacobian of the dynamics having real and non-positive eigenvalues. I will then describe a novel two point correlation result for general free products of the form M = O B O^T A for O sampled from the Haar measure. I will use this result to characterize the exact asymptotics of the performance of a linear transformer trained to perform in-context linear regression on “generic” (randomly rotated) covariance matrices.
URL:https://cmsa.fas.harvard.edu/event/member-seminar-101725/
LOCATION:Common Room\, CMSA\, 20 Garden Street\, Cambridge\, MA\, 02138\, United States
CATEGORIES:Member Seminar
ATTACH;FMTTYPE=image/png:https://cmsa.fas.harvard.edu/media/CMSA-Member-Seminar-10.17.25-scaled.png
END:VEVENT
END:VCALENDAR