Foundations of Machine Learning

ABOUT THE COURSE: This is a course on Machine Learning that introduces the subject in a formal way with a focus on basic and foundational concepts. A holistic view of machine learning is taken, including aspects of optimization (SGD) that are ignored in classical narratives. Topics that are relevant to modern models are highlighted so that the connection to state-of-the-art is not missed in the theoretical exploration. For example, in case of linear regression, apart from classical topics like bias-variance trade-off and regularization, an analysis in the overparametrization regime is also taught that immediately connects you to modern day complex models with billions of parameters. Finally, foundational topics for generative models are also covered, including energy-based models and Langevin sampling. INTENDED AUDIENCE: Students wishing to start research in core machine learning areas PREREQUISITES: Anyone with a good understanding of basic engineering math is eligible. Especially subjects like linear algebra, probability theory, and optimization. INDUSTRY SUPPORT: Microsoft, Google, Meta, Amazon, OpenAI