STA 290 Seminar Series
Tuesday, March 31st, 2015, 4:10pm, MSB 1147 (Colloquium Room)
Refreshments at 3:30pm in MSB 4110 (Statistics Lounge)
Speaker: Bala Rajaratnam (Stanford University)
Title: The Deterministic Bayesian Lasso: Estimation and Shrinkage via Limit of Gibbs Sampling
Abstract: The application of the lasso is popular in high-dimensional settings where only a small number of the regression coefficients are believed to be nonzero (i.e., the solution is sparse). Coordinatewise methods, the most common means of computing the lasso solution, naturally work well in the presence of low to moderate multicollinearity. The computational speed of coordinatewise algorithms degrades however as sparsity decreases or when multicollinearity increases. Lack of sparsity and high multicollinearity can be quite common in contemporary applications but model selection is often a necessity in such settings. Motivated by such limitations, we propose the novel “Deterministic Bayesian Lasso” algorithm. This algorithm is developed by considering a limiting version of the Bayesian lasso and demonstrates how a Bayesian perspective can yield important insights. In contrast to coordinatewise algorithms, the performance of the Deterministic Bayesian Lasso improves as sparsity decreases and multicollinearity increases. Importantly, in non-sparse and high-multicollinearity settings the proposed algorithm can offer substantial increases in computational speed over coordinatewise algorithms. A rigorous theoretical analysis demonstrates that (1) the Deterministic Bayesian Lasso algorithm converges to the lasso solution, and (2) it leads to a representation of the lasso estimator which shows how it achieves both ℓ1 and ℓ2 types of shrinkage simultaneously. Moreover, convergence can be obtained under more general assumptions. Connections between the Deterministic Bayesian Lasso and other approaches are also provided. The benefits of the Deterministic Bayesian Lasso algorithm are then illustrated on simulated and real data. (Joint work with S.Roberts, D.Sparks, & O. Dalal)