STA 142B Statistical Learning II

Units: 4


  • Lecture - 3.0 hours
  • Discussion - 1.0 hours

Catalog Description: Fundamental concepts and methods in statistical learning with emphasis on unsupervised learning. Principles, methodologies and applications of clustering methods, dimension reduction and manifold learning techniques, graphical models and latent variables modeling.

Prerequisite: STA 142A; (STA 130B or STA 131B))

Goals: Students learn how to use a variety of supervised statistical learning methods, and gain an understanding of their relative advantages and limitations. In addition to learning concepts and heuristics for selecting appropriate methods, the students will also gain programming skills in order to implement such methods. The students will also learn about the core mathematical constructs, optimization techniques and probabilistic algorithms behind the methods. A primary emphasis will be on understanding the methodologies through numerical simulations and analysis of real-world data. A high level programming language like R or Python will be used for the computation, and students will become familiar with using existing packages for implementing specific methods.

Summary of course contents:

  • Concepts of unsupervised learning
    • Challenges with unlabeled data
    • Dimensionality issues and phenomena
    • Relevant constrained optimization concepts
  • Linear dimensionality reduction
    • Principal components analysis (PCA)
    • Canonical correlation analysis (CCA)
    • Independent components analysis (ICA)
  • Clustering
    • K-means clustering
    • Hierarchical clustering
    • Spectral clustering
    • Practical limitations of clustering
  • Nonlinear dimensionality reduction and manifold learning
    • Concepts of data on manifolds
    • Multidimensional scaling
    • Kernel PCA
    • Local linear embedding, Laplacian and Hessian eigenmap
    • Graph-based learning and Isomap
  • Bayesian learning paradigm and latent variables modeling
    • Generative models for data
    • Prior and posterior
    • Gaussian process models
    • Dirichlet process models
    • Hidden Markov models
    • Overview of computational schemes
  • Graphical models
    • Undirected graphs and Gaussian Graphical Models (GGM)
    • Causal modeling and Directed Acyclic Graphs (DAG)
  • Latent variables models for discrete data
    • Loglinear models
    • Latent Dirichlet allocation (LDA)

Credit Restrictions: None

Illustrative reading:

  • An Introduction to Statistical Learning, with Applications in R – G. James, D. Witten, T.
    Hastie, R. Tibshirani
  • Modern Multivariate Statistical Techniques, 2nd Ed. -- A. J. Izenman
  • Machine Learning: A Probabilistic Perspective – K. P. Murphy and F. Bach


Potential Overlap:

Some of the broad topics, such as linear dimension techniques overlap with materials in STA
135. However, the emphasis in 135 is on understanding methods within the context of a statistical
model, and their mathematical derivations and broad application domains. In contrast, STA 142B
focuses more on issues of the statistical principles and optimization schemes inherent in the
formulation of the methods, their advantages and limitations, and their actual performance, as
evidenced by numerical simulations and data analysis. The computational component has some
overlap with STA 141C, where the emphasis is more on high performance computing and
implementation of optimization techniques. A portion of the course overlaps with content of STA
145. However, here the focus will be on conceptual description and implementation rather than
mathematical foundations of inferential and computational tools.

There is also some overlap of content of ECS 171, but less so as compared to STA 142A.
However, the emphasis there is more on the implementation of the techniques and their
applications. In addition, ECS 171 covers both unsupervised and supervised learning methods in
one course, whereas STA 142B is dedicated to unsupervised learning methods, and so will go
into technical and mathematical details at a greater depth.

First offering Spring 2020.