Click on the text like “Week 1: Sep 02 – Sep 06” to expand or collapse the items we covered in that week.

I will fill in more detail and provide links to lecture notes and labs as we go along. Items for future dates are tentative and subject to change.

Wed, Sep 04

Fri, Sep 06

  • In class, we will work on:
    • Polynomial Regression (Section 3.3 of ISLR): pdf
    • Motivate model selection by minimizing Residual Sum of Squares (Section 3.1 of ISLR): pdf
  • After class, please:
    • Continue reading Chapter 1, and Sections 2.1, 2.2.1, 2.2.2, 3.1, 3.2, and 3.3 of ISLR.
    • Start on Homework 1, due Friday Sep 13; assigned on GitHub.

Mon, Sep 09

  • In class, we will work on:
    • Multiple regression in matrix notation: pdf
    • Model estimation by minimizing Residual Sum of Squares (Sections 3.1.1 and 3.2.1 of ISLR)
      • Lecture notes: pdf. Note that our derivation in terms of matrices is not discussed in our text book. :(
  • After class, please:
    • Continue reading Chapter 1, and Sections 2.1, 2.2.1, 2.2.2, 3.1, 3.2, and 3.3 of ISLR.

Wed, Sep 11

  • In class, we will work on:
    • Finish second example from last class.
    • Review of orthogonal projections
      • Lecture notes: pdf
    • Matrix/linear algebra view of fitted values
      • \(H = X (X'X)^{-1} X'\) is the hat matrix.
      • \(\hat{y} = X \hat{\beta} = Hy\) is the orthogonal projection of y into the column space of \(X\).
      • ANOVA Example: html
  • After class, please:
    • Work on HW1, due Friday Sep 13

Fri, Sep 13

  • In class, we will work on:
    • R commands for doing matrix operations .R
    • What it means for X to not have full column rank
      • Handout: pdf
      • Interactive plots for first example: html
    • If time, start a worksheet that will be part of your next homework assignment: pdf
  • After class, please:
    • Do HW2, due Friday Sep 20, to be assigned on GitHub some time Saturday

Mon, Sep 16

  • In class, we will work on:
    • Train/test splits:
      • Motivating example: pdf
      • Example wrap up: pdf
    • Lab 01, about train/test splits
  • After class, please:
    • Read Sections 5.1.1 through 5.1.4 of ISLR
    • Work on HW2, due Friday Sep 20

Wed, Sep 18

Fri, Sep 20

  • In class, we will work on:
    • Pairs Plots: pdf
    • Work on labs
  • After class, please:

Mon, Sep 23

  • In class, we will work on:
    • Expected test set MSE and the Bias/Variance trade-off.
      • Slides: pdf
      • Source for Slides: Rmd
  • After class, please:
    • Finish Lab 2
    • Study for quiz on Wed

Wed, Sep 25

Fri, Sep 27

  • In class, we will work on:
    • Transformations: pdf
  • After class, please:

Mon, Sep 30

  • In class, we will work on:
  • After class, please:

Wed, Oct 02

Fri, Oct 04

  • In class, we will work on:
    • Measuring error rates and cross-validation for classification (continuing lecture notes from last class)
    • Start handout from last class
  • After class, please:

Mon, Oct 07

Wed, Oct 09

  • In class, we will work on:
    • Midterm
  • After class, please:

Fri, Oct 11

  • In class, we will work on:
    • Answer to problem 2 (a) and (b) on HW
  • After class, please:

Mon, Oct 14

  • No Class: Midsemester Break. Safe travels!

Wed, Oct 16

  • In class, we will work on:
    • Logistic Regression
      • Lecture notes: pdf
      • Handout: pdf
  • After class, please:

Fri, Oct 18

  • In class, we will work on:
    • Logistic Regression with multiple explanatory variables pdf
  • After class, please:

Mon, Oct 21

  • In class, we will work on:
    • Finish handout on logistic regression with multiple explanatory variables from last class.
    • Estimation for logistic regression: pdf
  • After class, please:

Wed, Oct 23

  • In class, we will work on:
    • Measures of classification skill: pdf
    • Work on Lab 5
  • After class, please:
    • Lab 5 due Fri Oct 25
    • HW 5 due Wed Oct 30

Fri, Oct 25

  • In class, we will work on:
    • Multinomial Logistic Regression and AIC
      • Lecture notes pdf
      • Handout about multnomial logistic regression pdf
  • After class, please:

Mon, Oct 28

  • In class, we will work on:
    • Penalized estimation
      • hand out: pdf
      • we also defined the LASSO regression estimator as minimizing RSS + \(\lambda \sum_{j = 1}^p \vert\beta_j\vert\), and ridge regression as minimizing RSS + \(\lambda \sum_{j=1}^p \beta_j^2\)
  • After class, please:
    • Work on HW due Wed Oct 30

Wed, Oct 30

  • In class, we will work on:
    • Classification and regression trees
      • lecture notes: pdf
      • hand out: pdf
  • After class, please:

Fri, Nov 01

  • In class, we will work on:
    • Quiz
    • Overview of where we’ve been and where we’re going: pdf
  • After class, please

Mon, Nov 04

  • In class, we will work on:
    • Start on ensembles and stacking:
      • Lecture notes: pdf
      • Stacking schematic: pdf
  • After class, please:

Wed, Nov 06

  • In class, we will work on:
    • Code for stacking:
      • Classification example: pdf
      • Regression example: pdf
  • After class, please:
    • Prepare for quiz on Friday

Fri, Nov 08

  • In class, we will work on:
    • Quiz on logistic regression and classification error
    • Start on bagging, feature subsets, and random forests
      • lecture notes: pdf
      • hand out: pdf
  • After class, please:

Mon, Nov 11

  • In class, we will work on:
    • Finish bagging, feature subsets, and random forests
  • After class, please:

Wed, Nov 13

  • In class, we will work on:
    • quiz on trees and penalized estimation
    • Start on gradient tree boosting
      • hand out: pdf
      • lecture notes: pdf
  • After class, please:

Fri, Nov 15

  • In class, we will work on:
  • After class, please:

Mon, Nov 18

  • In class, we will work on:
  • After class, please:

Wed, Nov 20

  • In class, we will work on:
  • After class, please:

Fri, Nov 22

  • In class, we will work on:
  • After class, please:

Mon, Nov 25

  • In class, we will work on:
  • After class, please:

Wed, Nov 27

  • No Class: Thanksgiving Break. Safe travels!

Fri, Nov 29

  • No Class: Thanksgiving Break. Safe travels!

Mon, Dec 02

  • In class, we will work on:
  • After class, please:

Wed, Dec 04

  • In class, we will work on:
    • Evan is away at a conference.
    • Time to work on your projects in groups.
  • After class, please:

Fri, Dec 06

  • In class, we will work on:
    • Evan is away at a conference.
    • Time to work on your projects in groups.
  • After class, please:

Mon, Dec 09

  • In class, we will work on:
  • After class, please:

We will not have a final exam in this class.