Course schedule
The titles name the broader topic in each lecture, and the questions identify the issue that organizes it. The 26 lectures run from September 1 through December 8. The September 22 class was cancelled. Fall Reading Week is November 9–13, with no classes. Readings and precise theorem selections will be added later.
| Week | Lecture A | Lecture B | Coursework milestone |
|---|---|---|---|
| 1 | Tue, Sep 1 · Lecture 1 Decoder transformers and autoregressive inference How does a language model turn a prefix into a next-token distribution and generate a sequence? Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:50 UTC-07:00 |
Thu, Sep 3 · Lecture 2 Learning, information, and structural generalization Why use learning, and what makes good behavior on unseen inputs possible? Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:50 UTC-07:00 |
Homework 0 readiness exercise; submit on Canvas by Sun, Sep 6, 11:59 p.m.; architecture reference: LLM Architecture Gallery |
| 2 | Tue, Sep 8 · Lecture 3 Probabilistic models, log loss, and compression Why do language models use log loss rather than another proper loss, and what does a reported log-loss value mean? Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:51 UTC-07:00 |
Thu, Sep 10 · Lecture 4 Achieving goals Is next-token prediction with successful examples sufficient to get successful behavior? When and to what extent? What lies beyond? Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:51 UTC-07:00 |
Homework 1 PDF · source bundle |
| 3 | Tue, Sep 15 · Lecture 5 The dream and reality of state-space models Why state-space models? What are the challenges to learn them? Lessons from system identification Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:51 UTC-07:00 |
Thu, Sep 17 · Lecture 6 Data collection, dependence, and useful evidence Beyond independence: how do collection, preprocessing, and dependence determine how much useful evidence a dataset provides? Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:50 UTC-07:00 |
Homework 1 due Fri, Sep 18, 11:59 p.m.; submit on Canvas; Homework 2 PDF · source bundle · submit on Canvas |
| 4 | Tue, Sep 22 Class cancelled No class. |
Thu, Sep 24 · Lecture 7 Distribution shift and helping models learn When does distribution shift hurt, and how can data or task design make learning possible?Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:46 UTC-07:00 |
Homework 2 in progress; research-question sketch encouraged |
| 5 | Tue, Sep 29 · Lecture 8 The need for signal Directly learning to perform some simple algorithm with noisy gradient descent is hard, but intermediate supervision can fix this.Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:46 UTC-07:00 |
Thu, Oct 1 · Lecture 9 Interpolation and benign overfitting Does fitting the training data, including noise, force poor generalization? Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:51 UTC-07:00 |
Homework 2 due Fri, Oct 2, 11:59 p.m.; submit on Canvas; Homework 3 PDF · source bundle · submit on Canvas |
| 6 | Tue, Oct 6 · Lecture 10 Implicit bias and solution selection When many predictors fit, which one does training select? Lecture notes: PDF · LaTeXLast revised: 2026-10-09 23:37 UTC-07:00 |
Thu, Oct 8 · Lecture 11 Global convergence from local gradients When is local gradient information sufficient for global learning? Lecture notes: PDF · LaTeXLast revised: 2026-10-12 22:27 UTC-07:00 |
Research-question first draft due Sun, Oct 11, 11:59 p.m.; submit on Canvas; Homework 3 in progress |
| 7 | Tue, Oct 13 · Lecture 12 Accelerated and adaptive first-order methods How can first-order descent be accelerated and better scaled? Lecture notes: PDF · LaTeXLast revised: 2026-10-25 21:25 UTC-06:00 |
Thu, Oct 15 · Lecture 13 Representation learning and transfer When can several tasks reveal a representation that helps a new task? Lecture notes: PDF · LaTeXLast revised: 2026-10-25 21:25 UTC-06:00 |
Homework 3 due Fri, Oct 16, 11:59 p.m.; submit on Canvas |
| 8 | Tue, Oct 20 · Lecture 14 Neural networks as computational models Can neural networks represent algorithms and symbolic computation? Lecture notes: PDF · LaTeXLast revised: 2026-10-25 23:07 UTC-06:00 |
Thu, Oct 22 · Lecture 15 Approximation by neural networks What functions can neural networks approximate, and when is approximation economical? Lecture notes: PDF · LaTeXLast revised: 2026-10-25 23:41 UTC-06:00 |
No coursework deadline |
| 9 | Tue, Oct 27 · Lecture 16 Harnesses, state, and repeated computation Do harnesses increase computational power? |
Thu, Oct 29 · Lecture 17 Verification-guided search When does verification turn generation into efficient search? |
Homework 4 PDF · source bundle · submit on Canvas; due Fri, Nov 6, 11:59 p.m. |
| 10 | Tue, Nov 3 · Lecture 18 Offline and interactive imitation Can log-loss behavior cloning avoid compounding error, and when does interaction help? |
Thu, Nov 5 · Lecture 19 Preference models and objectives What do pairwise preferences identify, and what are preference objectives optimizing? |
Homework 4 due Fri, Nov 6, 11:59 p.m.; submit on Canvas; Homework 5 in progress |
| — | Tue, Nov 10 Fall Reading Week No class. |
Thu, Nov 12 Fall Reading Week No class. |
No classes November 9–13; Homework 5 in progress |
| 11 | Tue, Nov 17 · Lecture 20 Optimizing learned feedback When does optimizing estimated feedback improve behavior instead of exploiting a proxy? |
Thu, Nov 19 · Lecture 21 Catastrophic forgetting and interference Why can a learner forget even when all tasks have a common solution? |
Homework 5 due Fri, Nov 20, 11:59 p.m.; Homework 6 released |
| 12 | Tue, Nov 24 · Lecture 22 Memory in continual learning When can the past be compressed into fixed memory without losing what future learning needs? |
Thu, Nov 26 · Lecture 23 Statistical and exact learning When does statistical success mean that the underlying rule has been learned? |
Revised research-question note due Sun, Nov 29, 11:59 p.m.; submit on Canvas; Homework 6 in progress |
| 13 | Tue, Dec 1 · Lecture 24 Denoising and score estimation Why does denoising reveal a distribution? |
Thu, Dec 3 · Lecture 25 Reverse diffusion and generative dynamics Why can reverse dynamics generate, and how do errors accumulate? |
Homework 6 due Fri, Dec 4, 11:59 p.m. |
| Final meeting | Tue, Dec 8 · Lecture 26 Transformer training on accelerators Why does transformer training map so well to modern accelerators? |
— | New mathematical material ends |
Topics also potentially include evaluation, uncertainty, contamination, distribution shift, empirical scaling laws, curricula, and synthetic data and these are incorporated where their mathematics arises.