Event Date
Speaker: Po-Ling Loh (Professor, Statistical Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge
Title: Differentially private M-estimation via noisy optimization
Abstract: We present a noisy composite gradient descent algorithm for differentially private statistical estimation in high dimensions. We begin by providing general rates of convergence for the parameter error of successive iterates under assumptions of local restricted strong convexity and local restricted smoothness. Our analysis is local, in that it ensures a linear rate of convergence when the initial iterate lies within a constant-radius region of the true parameter. At each iterate, multivariate Gaussian noise is added to the gradient in order to guarantee that the output satisfies Gaussian differential privacy. We then derive consequences of our theory for linear regression and mean estimation. Motivated by M-estimators used in robust statistics, we study loss functions which downweight the contribution of individual data points in such a way that the sensitivity of function gradients is guaranteed to be bounded, even without the usual assumption that our data lie in a bounded domain. We prove that the objective functions thus obtained indeed satisfy the restricted convexity and restricted smoothness conditions required for our general theory. We will also discuss the benefits of acceleration in optimization procedures, specifically a private version of the Frank-Wolfe algorithm, and its consequences for statistical estimation.
This is based on joint work with Marco Avella-Medina, Casey Bradshaw, Zheng Liu, and Laurentiu Marchis.
Bio: Po-Ling Loh received her PhD in Statistics from UC Berkeley in 2014. From 2014-2016, she was an Assistant Professor of Statistics at the University of Pennsylvania. From 2016-2018, she was an Assistant Professor of Electrical & Computer Engineering at UW-Madison, and from 2019-2020, she was an Associate Professor of Statistics at UW-Madison and a Visiting Associate Professor of Statistics at Columbia University. She began her appointment in the Department of Pure Mathematics and Mathematical Statistics at the University of Cambridge in 2021, where she is currently a Professor of Statistics and a Fellow of St. John’s College. Po-Ling's current research interests include high-dimensional statistics, robustness, and differential privacy. She is a recipient of a Philip Leverhulme Prize, NSF CAREER Award, ARO Young Investigator Award, IMS Tweedie New Researcher Award, Bernoulli Society New Researcher Award, a Leverhulme Prize, an Ethel Newbold Prize, and a Hertz Fellowship.
Web-page (links to Cambridge): https://www.dpmms.cam.ac.uk/~pll28/