BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CMSA - ECPv6.15.20//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://cmsa.fas.harvard.edu
X-WR-CALDESC:Events for CMSA
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20230312T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20231105T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20240310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20241103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20250309T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20251102T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20241004T120000
DTEND;TZID=America/New_York:20241004T130000
DTSTAMP:20260529T221337
CREATED:20240907T183353Z
LAST-MODIFIED:20240930T155114Z
UID:10003464-1728043200-1728046800@cmsa.fas.harvard.edu
SUMMARY:High-dimensional learning of narrow neural networks
DESCRIPTION:Member Seminar \nSpeaker: Hugo Cui\, CMSA \nTitle: High-dimensional learning of narrow neural networks \nAbstract: This talk explores the interplay between neural network architectures and data structure through the lens of high-dimensional asymptotics. We focus on a class of narrow neural networks\, namely networks possessing a finite number of hidden units\, while operating in high dimensions. In the limit of large data dimension and comparably large number of samples\, we derive a tight asymptotic characterization of the learning of these architectures. As an illustration\, we discuss how this characterization enables the analysis of a solvable model of dot-product attention. We show how the latter can learn to implement either a positional attention mechanism (with tokens attending to each other based on their respective positions)\, or a semantic attention mechanism (with tokens attending to each other based on their meaning)\, and evidence a phase transition with sample complexity from positional to semantic learning.
URL:https://cmsa.fas.harvard.edu/event/member-seminar-10424/
LOCATION:CMSA Room G10\, CMSA\, 20 Garden Street\, Cambridge\, MA\, 02138\, United States
CATEGORIES:Member Seminar
ATTACH;FMTTYPE=image/png:https://cmsa.fas.harvard.edu/media/CMSA-Member-Seminar-10.4.24.png
END:VEVENT
END:VCALENDAR