BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CMSA - ECPv6.17.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:CMSA
X-ORIGINAL-URL:https://cmsa.fas.harvard.edu
X-WR-CALDESC:Events for CMSA
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20190310T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20191103T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20200308T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20201101T060000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:20210314T070000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:20211107T060000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/New_York:20201028T150000
DTEND;TZID=America/New_York:20201028T160000
DTSTAMP:20240515T201157Z
CREATED:20240127T031359Z
LAST-MODIFIED:20240515T201157Z
UID:10001502-1603897200-1603900800@cmsa.fas.harvard.edu
SUMMARY:Generalization bounds for rational self-supervised learning algorithms\, or "Understanding generalizations requires rethinking deep learning"
DESCRIPTION:Speakers: Boaz Barak and Yamini Bansal\, Harvard University Dept. of Computer Science \nTitle: Generalization bounds for rational self-supervised learning algorithms\, or “Understanding generalizations requires rethinking deep learning” \nAbstract: The generalization gap of a learning algorithm is the expected difference between its performance on the training data and its performance on fresh unseen test samples. Modern deep learning algorithms typically have large generalization gaps\, as they use more parameters than the size of their training set. Moreover the best known rigorous bounds on their generalization gap are often vacuous. In this talk we will see a new upper bound on the generalization gap of classifiers that are obtained by first using self-supervision to learn a complex representation of the (label free) training data\, and then fitting a simple (e.g.\, linear) classifier to the labels. Such classifiers have become increasingly popular in recent years\, as they offer several practical advantages and have been shown to approach state-of-art results. We show that (under the assumptions described below) the generalization gap of such classifiers tends to zero as long as the complexity of the simple classifier is asymptotically smaller than the number of training samples. We stress that our bound is independent of the complexity of the representation that can use an arbitrarily large number of parameters. Our bound assuming that the learning algorithm satisfies certain noise-robustness (adding small amount of label noise causes small degradation in performance) and rationality (getting the wrong label is not better than getting no label at all) conditions that widely (and sometimes provably) hold across many standard architectures. We complement this result with an empirical study\, demonstrating that our bound is non-vacuous for many popular representation-learning based classifiers on CIFAR-10 and ImageNet\, including SimCLR\, AMDIM and BigBiGAN. The talk will not assume any specific background in machine learning\, and should be accessible to a general mathematical audience. Joint work with Gal Kaplun. \n 
URL:https://cmsa.fas.harvard.edu/event/10-28-2020-new-technologies-in-mathematics-seminar/
LOCATION:MA
CATEGORIES:New Technologies in Mathematics Seminar
ATTACH;FMTTYPE=image/png:https://cmsa.fas.harvard.edu/media/CMSA-New-Technologies-in-Mathematics-10.28.20.png
END:VEVENT
END:VCALENDAR