STA / BST 290 Seminar Series
Tuesday, January 27, 4:10pm, MSB 1147 (Colloquium Room)
Refreshments at 3:30pm in MSB 4110 (Statistics Lounge)
Speaker: Jascha Sohl-Dickstein (Stanford University)
Title: Training and Manipulating Large Scale Probabilistic Models
Abstract: As datasets become increasingly massive, and models become more complex, progress is often limited by our ability to train and manipulate large scale probabilistic models. I will present novel methods for defining, training, sampling from, and evaluating probabilistic models by combining and extending ideas from statistics, non-equilibrium physics, dynamical systems, and machine learning.
I will first focus on a technique, Minimum Probability Flow learning (MPF), for consistent and rapid parameter estimation in unnormalized probabilistic models. MPF functions by minimizing the flow of probability away from the data distribution under a deterministic process whose stationary distribution is the model distribution. It does not require computing an intractable normalization constant or expensive sampling from the model distribution.
In addition to MPF, I will briefly discuss several other projects which address challenges in large scale probabilistic modeling. These include a method for accelerating Monte Carlo sampling by abandoning detailed balance, an optimization algorithm which combines the benefits of quasi-Newton and stochastic gradient descent, and an approach to unsupervised deep learning based on time-reversed diffusion.