BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CMSA - ECPv6.17.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:CMSA
X-ORIGINAL-URL:https://cmsa.fas.harvard.edu
X-WR-CALDESC:Events for CMSA
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20210314T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20211107T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20220313T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20221106T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20230312T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20231105T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20220303T151100
DTEND;TZID=America/New_York:20220303T161100
DTSTAMP:20240301T104734Z
CREATED:20240215T091737Z
LAST-MODIFIED:20240301T104734Z
UID:10002711-1646320260-1646323860@cmsa.fas.harvard.edu
SUMMARY:Towards Understanding Training Dynamics for Mildly Overparametrized Models
DESCRIPTION:Abstract: While over-parameterization is widely believed to be crucial for the success of optimization for the neural networks\, most existing theories on over-parameterization do not fully explain the reason — they either work in the Neural Tangent Kernel regime where neurons don’t move much\, or require an enormous number of neurons. In this talk I will describe our recent works towards understanding training dynamics that go beyond kernel regimes with only polynomially many neurons (mildly overparametrized). In particular\, we first give a local convergence result for mildly overparametrized two-layer networks. We then analyze the global training dynamics for a related overparametrized tensor model. For both works\, we rely on a key intuition that neurons in overparametrized models work in groups and it’s important to understand the behavior of an average neuron in the group. Based on two works: https://arxiv.org/abs/2102.02410 and https://arxiv.org/abs/2106.06573. \nBio: Professor Rong Ge is Associate Professor of Computer Science at Duke University. He received his Ph.D. from the Computer Science Department of Princeton University\, supervised by Sanjeev Arora. He was a post-doc at Microsoft Research\, New England. In 2019\, he received both a Faculty Early Career Development Award from the National Science Foundation and the prestigious Sloan Research Fellowship. His research interest focus on theoretical computer science and machine learning. Modern machine learning algorithms such as deep learning try to automatically learn useful hidden representations of the data. He is interested in formalizing hidden structures in the data and designing efficient algorithms to find them. His research aims to answer these questions by studying problems that arise in analyzing text\, images\, and other forms of data\, using techniques such as non-convex optimization and tensor decompositions.
URL:https://cmsa.fas.harvard.edu/event/3-3-2022-interdisciplinary-science-seminar/
CATEGORIES:Interdisciplinary Science Seminar
ATTACH;FMTTYPE=image/jpeg:https://cmsa.fas.harvard.edu/media/CMSA-Interdisciplinary-Science-Seminar-03.03.2022-1583x2048-1.jpg
END:VEVENT
END:VCALENDAR