Distributed Optimization and Machine Learning

ABOUT THE COURSE: Centralized access to information and its subsequent processing is often computationally prohibitive over large networks due to communication overhead and the scale of the problem. Consequently, such systems rely on control and optimization algorithms that are fully distributed or even decentralized in nature. This course will provide a comprehensive overview of design and analysis of distributed optimization algorithms and their applications to machine learning. The aim is to revisit classical control and optimization algorithms for centralized optimization and discuss how these can be extended to distributed setting to accommodate the effects of communication constraints, network topology, computational resources, and robustness. Topics include graph theory, iterative methods for convex problems, synchronous and asynchronous setups, consensus algorithms, and distributed machine learning. We will also explore some recent literature in this area that exploits control theory for design of accelerated distributed optimization algorithms. INTENDED AUDIENCE: MTech and PhD students in broad areas of optimization and data science PREREQUISITES: A background in convex optimization and differential equations is preferred INDUSTRY SUPPORT: Tata Consultancy Services, Microsoft, Google, Amazon, IBM, etc.