More on Kubota’s spectrum of invertible states

Speaker: Carl Allen and Ivana Balažević - University of Edinburgh School of Informatics Title: Knowledge graph representation: From recent models towards a theoretical understanding Abstract: Knowledge graphs (KGs), or knowledge bases, are large repositories of facts in the form of triples (subject, relation, object), e.g. (Edinburgh, capital_of, Scotland). Many models have been developed to succinctly represent KGs […]

https://youtu.be/OoimTbnSe7I Speaker: Nikunj Saunshi, Dept. of Computer Science, Princeton University Title: A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks Abstract: Autoregressive language models pretrained on large corpora have been successful at solving downstream tasks, even with zero-shot usage. However, there is little theoretical justification for their success. This paper considers the following […]