BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CMSA - ECPv6.15.20//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://cmsa.fas.harvard.edu
X-WR-CALDESC:Events for CMSA
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20220313T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20221106T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20230312T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20231105T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20240310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20241103T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20231201T120000
DTEND;TZID=America/New_York:20231201T130000
DTSTAMP:20260602T022255
CREATED:20240227T112716Z
LAST-MODIFIED:20240227T112731Z
UID:10002878-1701432000-1701435600@cmsa.fas.harvard.edu
SUMMARY:An introduction to mixture of experts in deep learning
DESCRIPTION:Member Seminar \nSpeaker: Samy Jelassi \nTitle: An introduction to mixture of experts in deep learning \nAbstract: Scale has opened new frontiers in natural language processing – but at a high cost. Mixture-of-Experts (MoE) have been proposed as a path to even larger and more capable language models. They select different parameters for each incoming example. By doing so\, the parameter count is decoupled from the compute per example leading to very large\, but efficient models. In this talk\, I will review the concept of mixture of experts\, provide a basic description of the Switch Transformers model\, characterize some of their behaviors and conclude by highlighting some open problems in the field. This talk is mainly based on the following papers: https://arxiv.org/pdf/2101.03961.pdf\, https://arxiv.org/pdf/2209.01667.pdf .
URL:https://cmsa.fas.harvard.edu/event/member-seminar-12123/
LOCATION:CMSA Room G10\, CMSA\, 20 Garden Street\, Cambridge\, MA\, 02138\, United States
CATEGORIES:Member Seminar
END:VEVENT
END:VCALENDAR