How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad
CMSA Room G10 CMSA, 20 Garden Street, Cambridge, MA, United Stateshttps://youtu.be/C6NDdnSaluU New Technologies in Mathematics Seminar Speaker: Aryo Lotfi (EPFL) Title: How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad Abstract: Can Transformers predict new syllogisms by composing established ones? More generally, what type of targets can be learned by such models from scratch? Recent works show that Transformers can be Turing-complete in terms of […]