BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CMSA - ECPv6.17.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:CMSA
X-ORIGINAL-URL:https://cmsa.fas.harvard.edu
X-WR-CALDESC:Events for CMSA
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20200308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20201101T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20210314T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20211107T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20220313T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20221106T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20211201T140000
DTEND;TZID=America/New_York:20211201T143000
DTSTAMP:20240515T203641Z
CREATED:20230808T175251Z
LAST-MODIFIED:20240515T203641Z
UID:10001200-1638367200-1638369000@cmsa.fas.harvard.edu
SUMMARY:The Principles of Deep Learning Theory
DESCRIPTION:Speaker: Dan Roberts\, MIT & Salesforce \nTitle: The Principles of Deep Learning Theory \nAbstract: Deep learning is an exciting approach to modern artificial intelligence based on artificial neural networks. The goal of this talk is to provide a blueprint — using tools from physics — for theoretically analyzing deep neural networks of practical relevance. This task will encompass both understanding the statistics of initialized deep networks and determining the training dynamics of such an ensemble when learning from data. \nIn terms of their “microscopic” definition\, deep neural networks are a flexible set of functions built out of many basic computational blocks called neurons\, with many neurons in parallel organized into sequential layers. Borrowing from the effective theory framework\, we will develop a perturbative 1/n expansion around the limit of an infinite number of neurons per layer and systematically integrate out the parameters of the network. We will explain how the network simplifies at large width and how the propagation of signals from layer to layer can be understood in terms of a Wilsonian renormalization group flow. This will make manifest that deep networks have a tuning problem\, analogous to criticality\, that needs to be solved in order to make them useful. Ultimately we will find a “macroscopic” description for wide and deep networks in terms of weakly-interacting statistical models\, with the strength of the interactions between the neurons growing with depth-to-width aspect ratio of the network. Time permitting\, we will explain how the interactions induce representation learning. \nThis talk is based on a book\, The Principles of Deep Learning Theory\, co-authored with Sho Yaida and based on research also in collaboration with Boris Hanin. It will be published next year by Cambridge University Press.
URL:https://cmsa.fas.harvard.edu/event/12-1-21-new-technologies-in-mathematics-seminar-series/
LOCATION:Virtual
CATEGORIES:New Technologies in Mathematics Seminar
ATTACH;FMTTYPE=image/png:https://cmsa.fas.harvard.edu/media/CMSA-NTM-Seminar-12.01.21.png
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20211208T140000
DTEND;TZID=America/New_York:20211208T150000
DTSTAMP:20240515T203917Z
CREATED:20230808T175752Z
LAST-MODIFIED:20240515T203917Z
UID:10001201-1638972000-1638975600@cmsa.fas.harvard.edu
SUMMARY:Hierarchical Transformers are More Efficient Language Models
DESCRIPTION:Speaker: Piotr Nawrot\, University of Warsaw \nTitle: Hierarchical Transformers are More Efficient Language Models \nAbstract: Transformer models yield impressive results on many NLP and sequence modeling tasks. Remarkably\, Transformers can handle long sequences which allows them to produce long coherent outputs: full paragraphs produced by GPT-3 or well-structured images produced by DALL-E. These large language models are impressive but also very inefficient and costly\, which limits their applications and accessibility. We postulate that having an explicit hierarchical architecture is the key to Transformers that efficiently handle long sequences. To verify this claim\, we first study different ways to upsample and downsample activations in Transformers so as to make them hierarchical. We use the best performing upsampling and downsampling layers to create Hourglass – a hierarchical Transformer language model. Hourglass improves upon the Transformer baseline given the same amount of computation and can yield the same results as Transformers more efficiently. In particular\, Hourglass sets new state-of-the-art for Transformer models on the ImageNet32 generation task and improves language modeling efficiency on the widely studied enwik8 benchmark.
URL:https://cmsa.fas.harvard.edu/event/12-8-21-new-technologies-in-mathematics/
LOCATION:Virtual
CATEGORIES:New Technologies in Mathematics Seminar
ATTACH;FMTTYPE=image/png:https://cmsa.fas.harvard.edu/media/CMSA-NTM-Seminar-12.08.21.png
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20211215T140000
DTEND;TZID=America/New_York:20211215T150000
DTSTAMP:20240515T204057Z
CREATED:20230808T180208Z
LAST-MODIFIED:20240515T204057Z
UID:10001202-1639576800-1639580400@cmsa.fas.harvard.edu
SUMMARY:Unreasonable effectiveness of the quantum complexity view on quantum many-body physics
DESCRIPTION:Speaker: Anurag Anshu\, Department of EECS & Challenge Institute for Quantum Computation\, UC Berkeley \nTitle: Unreasonable effectiveness of the quantum complexity view on quantum many-body physics \nAbstract: A central challenge in quantum many-body physics is to estimate the properties of natural quantum states\, such as the quantum ground states and Gibbs states. Quantum Hamiltonian complexity offers a computational perspective on this challenge and classifies these natural quantum states using the language of quantum complexity classes. This talk will provide a gentle introduction to the field and highlight its success in pinning down the hardness of a wide variety of quantum states. In particular\, we will consider the gapped ground states and Gibbs states on low dimensional lattices\, which are believed to exhibit ‘low complexity’ due to the widely studied area law behaviour. Here\, we will see the crucial role of complexity-theoretic methods in progress on the ‘area law conjecture’ and in the development of efficient algorithms to classically simulate quantum many-body systems.
URL:https://cmsa.fas.harvard.edu/event/12-15-21-new-technologies-in-mathematics/
LOCATION:Virtual
CATEGORIES:New Technologies in Mathematics Seminar
ATTACH;FMTTYPE=image/png:https://cmsa.fas.harvard.edu/media/CMSA-NTM-Seminar-12.15.21-2.png
END:VEVENT
END:VCALENDAR