Research
My research interests are in reliable optimization methods for modern machine learning, especially stochastic methods, distributed systems, robustness, privacy, and variational inequalities.
For my complete filterable list of papers, see Publications; for my presentation materials, see Talks.
Research map
The ultimate goal of my research is to bridge the gap between theory and practice in Optimization in Machine Learning.
Distributed learning
Reliability and trust
Modern assumptions and algorithmic tools
Representative papers
A few papers that give a quick entry point into my main research themes.
On the Role of Batch Size in Stochastic Conditional Gradient Methods
Theoretically derived scaling laws that help tune batch size and stepsize in large-scale stochastic conditional gradient training.
High-Probability Bounds for the Last Iterate of Clipped SGD
Recent work on last-iterate convergence for clipped stochastic gradient methods under heavy-tailed noise.
Methods for Convex $(L_0,L_1)$-Smooth Optimization
Clipping, acceleration, and adaptivity for generalized smoothness assumptions beyond the classical Lipschitz-gradient model.
High-Probability Convergence for Composite and Distributed Optimization
Clipping stochastic gradient differences for composite, distributed, and variational inequality problems with heavy-tailed noise.
Variance Reduction Is an Antidote to Byzantines
The first variance-reduced Byzantine-robust method with strong convergence guarantees, weaker assumptions, and communication compression.
Secure Distributed Training at Scale
The first scalable Byzantine-robust distributed training method with convergence guarantees matching Byzantine-free SGD.
Extragradient Method: $O(1/K)$ Last-Iterate Convergence
Resolved a long-standing open question on last-iterate convergence of extragradient for monotone variational inequalities.
Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping
The first near-optimal high-probability complexity guarantees under heavy-tailed gradient noise assumptions.