Event Date
SPEAKER: Mina Karzand, Assistant Professor, Department of Statistics, UC Davis
TITLE: "Representation Costs of Linear Neural Networks: Analysis and Design"
ABSTRACT:
For different parameterizations (mappings from parameters to predictors), we study the regularization cost in predictor space induced by l_2 regularization on the parameters (weights). We focus on linear neural networks as parameterizations of linear predictors and identify the representation cost of certain sparse linear ConvNets and residual networks. In order to get a better understanding of how the architecture and parameterization affect the representation cost, we also study the reverse problem, identifying which regularizers on linear predictors (e.g., l_p quasi-norms, group quasi-norms, the k-support-norm, elastic net) can be the representation cost induced by simple l_2 regularization, and designing the parameterizations that do so.
To put the main work in context, we spend some time in the talk to review some recent results on generalization in overparameterized models. This review includes some topics that will be covered in STA 250 in the next Winter quarter.
DATE: Thursday October 28th, 2021
LOCATION: MSB 1147, Colloquium Room*
*This will be an in-person seminar and is open to the public. STA 290 registrants are required to attend in person. For others, there will be a Zoom link available if you choose to listen in to the seminar remotely. To access the Zoom meeting for this seminar, please contact the instructor Professor Drake or Pete Scully ([email protected]) for the meeting ID and password, stating your affiliation.