For a time, it was common to dismiss large language models (LLMs) as "just autocomplete" and therefore fundamentally incapable of sophisticated reasoning. Yet modern LLMs exhibit capabilities that would have seemed impossible only a few years ago, the latest of which is disproving a longstanding conjecture of Erdős on the unit distance problem. One might attribute this progress to larger models and larger training corpora, but this is far from the whole story: after one has already scraped most of the available high-quality text on the internet, obtaining substantially more training data becomes challenging. Instead, much of the recent progress has come from innovations in how LLMs are trained. In this talk, we will trace the evolution of post-training and inference procedures (RLHF, DPO, RLVR, test-time compute, etc.) that have transformed LLMs into much more than mere autocomplete.
