STA 290 Seminar: Julie Novak

Event Date

Mathematical Sciences 1147 (Colloquium Room)

SPEAKER: Julie Novak; Netflix

TITLE:  “Statistical Methodologies in Streaming Experimentation at Netflix”

ABSTRACT: This talk will begin with an overview of how controlled randomized tests assist in informed data-driven decision making at Netflix. I will then go over streaming Quality of Experience and the components necessary to improve it. I plan to dive into two specific challenges associated with experiments in this space and describe the statistical solutions that we use to address them. First, the metrics we are comparing often have very ‘non-normal’ distributions and extreme outliers, so t-tests and standard non-parametric tests have undesirable properties. We develop a suite of bootstrapping tools which provide practical and statistical significance, as well as easy to interpret visualizations. Second, we need a fast, automated failure detection system to ensure that tests do not break any devices. To solve this we built a Bayesian regression model which updates its decision as data streams come in.


DATE:                    Thursday, October 24th, 4:10pm

LOCATION:          MSB 1147, Colloquium Room

REFRESHMENTS: 3:30pm MSB 4110 (4th floor lounge)

STA 290 Seminar Series