Depen Morwani defends his PhD
Congratulations to Depen Morwani, one of the first PhD students from the group, on successfully defending his thesis. Onward.
people
Harvard Machine Learning Foundations Group
The Harvard Machine Learning Foundations Group studies the theoretical and empirical foundations of machine learning. The group is led by Sham Kakade, David Alvarez-Melis, and Boaz Barak, alongside affiliated faculty across computer science, applied mathematics, and statistics.
We are broadly interested in the foundations of deep learning. Active research spans optimization, generalization, reinforcement learning, generative modeling, and the algorithmic study of large language models.
We are affiliated with the Kempner Institute at Harvard.
Principal investigators
Affiliated faculty
Research themes
Each links to recent publications on that topic.
Theory of learning and generalization
Why over-parameterized models generalize, the role of inductive biases, sample complexity, and the limits of statistical learning.
Optimization for deep learning
Second-order methods, learning-rate schedules, weight decay, and the geometry of training in large models.
Scaling, emergence, and the empirical study of deep learning
Treating large models as empirical objects. Scaling laws, capability emergence, and the dynamics of pretraining.
Sampling and generative modeling
Diffusion, flow matching, masked diffusion, and the algorithmic foundations of generation.
Architectures, attention, and in-context learning
What transformers can compute, length generalization, the structure of attention, and the dynamics of in-context learning.
Robust and trustworthy machine learning
Distribution shift, conformal inference, interpretability, alignment, and the social context of machine learning.
Recent updates
Selected publications
A Simplified Analysis of SGD for Linear Regression with Weight Averaging
Alexandru Meterez, Depen Morwani, Costin-Andrei Oncescu, Jingfeng Wu, Cengiz Pehlevan, Sham Kakade · arXiv 2025
Any-Order Flexible Length Masked Diffusion
Jaeyeon Kim, C. Lee, Carles Domingo-Enrich, Yilun Du, Sham Kakade, Timothy Ngotiaoco, Sitan Chen, M. Albergo · arXiv 2025
Cognitive models can reveal interpretable value trade-offs in language models
Sonia K. Murthy, Rosie Zhao, Jennifer Hu, Sham Kakade, Markus Wulfmeier, Peng Qian, Tomer D. Ullman · arXiv 2025
Random Scaling of Emergent Capabilities
Rosie Zhao, Tian Qin, David Alvarez-Melis, Sham Kakade, Naomi Saphra · arXiv 2025
SOAP: Improving and Stabilizing Shampoo using Adam
Nikhil Vyas, Depen Morwani, Rosie Zhao, Itai Shapira, David Brandfonbrener, Lucas Janson, Sham Kakade · arXiv 2024
Universal Length Generalization with Turing Programs
Kaiying Hou, David Brandfonbrener, Sham Kakade, Samy Jelassi, Eran Malach · arXiv 2024
Joining the group
We are recruiting at every level. To get in touch, see the personal websites of the professors you'd like to work with. More details on the join page.