Theoretical Foundations in Modern Machine Learning
CSCI-GA.3033-141 · Fall 2026
Instructor: Allen Liu
Meeting time: Tuesdays, 4:55–6:55 PM
Location: Bobst LL138
Course Description
This class aims to showcase various ways that theoretical insights play a role in advancing modern machine learning. Topics include scaling laws, data and compute allocation, post-training and reinforcement learning, and inference-time interventions. We will emphasize theoretical perspectives that both help us understand modern training pipelines and inform new algorithms and design principles.
Course Information
Schedule
| Meeting | Theme | Detailed topic | Recommended readings |
|---|---|---|---|
| 1 | LLM fundamentals | ||
| 2 | LLM fundamentals | ||
| 3 | Pre-training | ||
| 4 | Pre-training | ||
| 5 | Pre-training | ||
| 6 | Post-training | ||
| 7 | Post-training | ||
| 8 | Post-training | ||
| 9 | Post-training | ||
| 10 | Test-time scaling | ||
| 11 | Test-time scaling | ||
| 12 | Test-time scaling | ||
| 13 | Flex topic | ||
| 14 | Flex topic |