Click on the text like “Week 1: Jan 20 – 24” to expand or collapse the items we covered in that week.

I will fill in more detail and provide links to lecture notes and labs as we go along. Items for future dates are tentative and subject to change.

#### Wed, Jan 22

• In class, we will work on:
• Discuss syllabus, overview of course
• Review of prerequisites
• Topics from probability: pdf (also posted on resources page)
• Practice examples for probability: pdf Solutions: pdf
• Common probability distributions: pdf (also posted on resources page; you don’t need to memorize this material)
• Topics from calculus (not discussed today, please review after class): pdf (also posted on resources page)
• After class, please do the following:
• Register for GitHub here if you haven’t already; I will ask you to provide your GitHub user name in the questionairre below.
• Fill out a brief questionnairre (if you are taking two classes with me, you only need to fill out this questionairre once)
• Fill out this brief poll about when my office hours should be held (if you are taking two classes with me, you only need to fill out this poll once)
• Review the prerequisite material from probability and calculus. If there are any topics from probability that you don’t have memorized, please start memorizing them. This will make the rest of this class go much more smoothly for you. If you have any questions on this material, please don’t hesitate to ask me in office hours or on Piazza. I am happy to help with this material.
• Over the next 1 or 2 classes we will cover material from Chapter 6 of the text. Give those sections in the book a brief skim.
• Homework 0 due 5 pm Mon, Jan 27

#### Fri, Jan 24

• In class, we will work on:
• Wrap up any remaining probability examples from last class
• Define the $$t$$, $$F$$, and $$\chi^2$$ distributions; this material is in Chapter 6 of Rice. Lecture notes: pdf Note that there is a minor error in the lecture notes that we did not make in class: on page 5, the denominator should be the square root of a chi squared random variable divided by its degrees of freedom. Basically, we need to take the square root of the square of the denominator to see that we have a t-distributed random variable.
• Lab 01. See labs page for solutions.
• Reading: Over the next few classes we will cover material from Sections 8.1, 8.2, 8.3, and 8.5 of the text. Give those sections in the book a brief skim.
• Homework 0 due 5 pm Mon, Jan 27

#### Mon, Jan 27

• In class, we will work on:
• Start on maximum likelihood estimation.
• Warm up example/worksheet: pdf
• Plots to go with the example above: pdf
• Lecture notes for maximum likelihood estimation for this example: pdf
• We had a few extra minutes so we also defined an estimator (a random variable) and estimate (realized value of an estimator based on a sample, a number).
• Reading: Over the next few classes we will cover material from Sections 8.1, 8.2, 8.3, and 8.5 of the text. Give those sections in the book a brief skim.
• Homework 0 due 5 pm today, Mon, Jan 27

#### Wed, Jan 29

• In class, we will work on:
• Quiz on probability background. This will be very similar to the examples from Wednesday Jan 22 and HW 0. Please memorize everything on the “Topics from probability” document linked on the Resources page, and know how to use it. You only need to know the informal statement of the Central Limit Theorem.
• More on maximum likelihood estimation:
• Example about seedlings: pdf Solutions: pdf
• Lab 2. Solutions: GitHub
• Reading: Continue working through Section 8.5. Also give Section 8.4 a brief skim.
• Homework 1 due 5 pm Wed, Feb 5

#### Fri, Jan 31

• In class, we will work on:
• More on maximum likelihood estimation:
• Example about bird hops: pdf
• Homework 1 due 5 pm Wed, Feb 5

#### Mon, Feb 03

• In class, we will work on:
• Example where analytically maximizing the likelihood doesn’t work: pdf
• Method of moments. Lecture notes: pdf
• Reminder about Taylor’s theorem, from calculus review handout (linked on resources page)
• Intro to Newton’s method for optimization. Slides: pdf
• Homework 1 due 5 pm Wed, Feb 5
• Reading: Over the next few days, we’ll explore using Newton’s method to numerically optimize the log-likelihood function. Wikipedia is a reasonably good source; you could give the following a skim:

#### Wed, Feb 05

• In class, we will work on:
• Newton’s method for numerical maximization of the log-likelihood function
• Slides posted Monday
• Start Zero-inflated Poisson example: html
• Homework 1 due 5 pm today Wed, Feb 5

#### Fri, Feb 07

• In class, we will work on:
• Quiz on maximum likelihood. I’ll ask you to find the maximum likelihood estimator for a model with 1 unknown parameter.
• Newton’s method for numerical maximization of the log-likelihood function
• Finish Zero-inflated Poisson example we started last class.

#### Mon, Feb 10

• In class, we will work on:
• Numerical maximum likelihood using Stan
• Handout: pdf
• Lab 3
• Homework 2 due 5pm Friday Feb 14

#### Wed, Feb 12

• In class, we will work on:
• Sampling distributions, bias, variance, and mean squared error. Lecture notes: pdf
• Homework 2 due 5pm Friday Feb 14

#### Fri, Feb 14

• In class, we will work on:
• Example of bias, variance, MSE for estimators of the variance of a normal distribution. Lecture notes: pdf
• We didn’t have time to get to Lab 4, but maybe you could take a quick look at it on your own?
• Motivation for subjective probability intervals: pdf
• Homework 2 due 5pm today Friday Feb 14

#### Mon, Feb 17

• In class, we will work on:
• Wrap up lab on bias, variance, and MSE
• Wrap up subjective probability intervals/subjective interpretation of probability
• Intro to Bayesian inference
• Lecture notes on Binomial example: pdf
• We also did the first example on this worksheet: pdf

#### Wed, Feb 19

• In class, we will work on:
• More on Bayesian inference:
• Lab 6
• Finish examples from practice worksheet last class.
• Homework 3 due Wed. Feb 26

#### Fri, Feb 21

• In class, we will work on:
• Bayesian credible intervals: posterior percentiles, highest posterior density. Partial lecture notes are here, but I also discussed highest posterior density: pdf
• Start on analysis of normal distribution from a Bayesian perspective.
• Lecture notes: pdf. Note that for case 1 (unknown mean, known variance) I mixed up the notation a little towards the end of page 1 of these notes. The stuff I wrote in class was correct.
• Plots illustrating MSE for Bayesian estimator of the mean when the variance is known: pdf
• Another practice example we didn’t have time for: pdf. Solutions: pdf
• Homework 3 due Wed. Feb 26

#### Mon, Feb 24

• In class, we will work on:
• Finish examples of estimation for normal distribution from last class.
• Introduce Monte Carlo integration:
• Lecture notes: pdf
• Examples: pdf
• Homework 3 due Wed. Feb 26

#### Wed, Feb 26

• In class, we will work on:
• Finish examples from last class
• Homework 3 due Thursday Feb. 27

#### Fri, Feb 28

• In class, we will work on:
• Some additional Monte Carlo integration examples: pdf
• Introduce MCMC

#### Mon, Mar 02

• In class, we will work on:
• Finish MCMC (correction to last pages of handout from last class) and lab on MCMC:
• Quiz 3

#### Wed, Mar 04

• In class, we will work on:
• Overview of next week or two: pdf
• Fisher information
• Motivation: pdf
• Lecture notes: pdf
• Connection to Taylor series: pdf
• If you want reading for the material we’ll cover in the next few days, you can refer to the following sources (in order of how useful I think they are):

#### Fri, Mar 06

• In class, we will work on:
• Large-sample approximation to sampling distribution of MLE
• Hand out with proof: pdf
• Hand out with conditions: pdf

#### Mon, Mar 09

• In class, we will work on:
• Large-sample normal approximation to sampling distribution of MLE (Continuing handouts from last class)

#### Wed, Mar 11

• In class, we will work on:
• Consistency and Efficiency
• Notes: pdf
• Set up for lab 10: pdf

#### Fri, Mar 13

• In class, we will work on:
• No new material.

#### Mon, Mar 16

• No Class: Midsemester Break. Safe travels!

#### Wed, Mar 18

• No Class: Midsemester Break. Safe travels!

#### Fri, Mar 20

• No Class: Midsemester Break. Safe travels!

#### Mon, Mar 23

• No Class: Midsemester Break. Safe travels!

#### Wed, Mar 25

• No Class: Midsemester Break. Safe travels!

#### Fri, Mar 27

• No Class: Midsemester Break. Safe travels!

#### Mon, Mar 30

• In class, we will work on:
• Reminder/overview of recent topics:
• Notes: pdf
• Errors:
• At about 16:50 in the video, I should have written $$\hat{\Theta}^{MLE}$$
• Statement of Theorem: Normal approximation to posterior distribution for large $$n$$.
• Notes: pdf
• Errors:
• At about minute 4:00 in the video, I incorrectly state that the marginal posterior distribution for the mean in a Bayesian analysis of a normal distribution with unknown variance is normal.
• Proof Part 1: Prior doesn’t matter:
• Proof Part 2: 2nd order Taylor approximation:

#### Wed, Apr 01

• In class, we will work on:
• Example of finding a normal approximation to the posterior distribution and using that approximation to find a credible interval:
• Worksheet and lab:
• Worksheet (written part introducing lab 11 – do this before lab 11)
• Written part: pdf
• Solutions to written part: pdf
• Now go on to lab 11, distributed via GitHub. Solutions will be posted on the labs page some time on Wednesday April 1.
• Finish worksheet and lab from today.

#### Fri, Apr 03

• In class, we will work on:
• Taking stock of where we are in the class:
• Motivating example for confidence intervals:
• Defining confidence intervals, and an example of a confidence interval derivation:
• Notes: pdf
• Errors:
• At about minute 36 I wrote down the wrong formula for the confidence interval. The formula should be $$[\bar{x} - t_{n-1}(1 - \frac{\alpha}{2}) \frac{s}{\sqrt{n}}, \bar{x} - t_{n-1}(\frac{\alpha}{2}) \frac{s}{\sqrt{n}}]$$
• The videos were longer than I originally planned, so we’ll do the example/lab next class.

#### Mon, Apr 06

• In class, we will work on:
• More details about confidence intervals: pivotal quantities, one-sided intervals
• Additional confidence interval example (variance of a normal distribution):
• Written part: pdf
• Written part solutions: pdf
• Lab 12

#### Wed, Apr 08

• In class, we will work on:
• Motivation for getting confidence intervals based on a large-sample normal approximation to a sampling distribution:
• Example of getting confidence intervals based on a large-sample normal approximation to a sampling distribution:
• Notes: pdf
• Results from R: pdf
• Example for you to do: Rayleigh distribution
• Written part: pdf
• Written part solutions: pdf
• Lab 13

#### Fri, Apr 10

• In class, we will work on:
• There is a little more content here than would fit into one class day. I’ll do a little less on Monday.
• Introduction to bootstrap for estimating a sampling distribution
• Annotated notes: pdf
• Example of bootstrap for estimating the distribution of the mean for a Poisson distribution
• Annotated notes: pdf
• Handout from video 1 with new notes I made for this video: pdf
• R Markdown document for example: Rmd GitHub

#### Mon, Apr 13

• In class, we will work on bootstrap confidence intervals
• Materials used throughout all videos today and next class
• Handout without my writing on it: pdf
• Handout with my notes on it: pdf
• R markdown file used to generate handout: Rmd GitHub
• Introduction and set up:
• Errors and Notes:
• At about 6:40, I gave an incorrect formula for the estimated standard error of the sample mean. The correct formula is $$\widehat{SE}(\bar{X}) = S/\sqrt{n} = \left[ \frac{\sum_{i=1}^n (X_i - \bar{X})^2}{n - 1} \right]^{0.5} / \sqrt{n}$$
• As I made this video, I claimed we really needed a bootstrap for this case. As I looked at the example more, I decided that although I’m most comfortable with a bootstrap based interval in this example, probably other approaches would also work OK.
• Percentile bootstrap confidence intervals:
• Bootstrap t confidence intervals when a formula is available to calculate the estimated standard error.

#### Wed, Apr 15

• In class, we will work on:
• Continuing with handout from April 13
• Bootstrap t confidence intervals when no formula is available to calculate the estimated standard error. Part 1: concepts
• Bootstrap t confidence intervals when no formula is available to calculate the estimated standard error. Part 2: code walk-through
• Lab on bootstrap confidence intervals

#### Fri, Apr 17

• In class, we will work on:
• Notes on worksheet from 2020-03-04. We talked about this handout in more detail that day. pdf
• First examples (more to come on Monday):

#### Mon, Apr 20

• In class, we will work on:
• Warm up example for likelihood ratio tests. I suggest giving the problem a try before watching the solutions video or looking at the solutions pdf.
• Warm up example: pdf
• Solutions: pdf
• Solutions video:
• Example of likelihood ratio tests for a normal distribution. I suggest giving the problem a try before watching the solutions video or looking at the solutions pdf.
• Example: pdf
• Solutions: pdf
• Solutions video:
• Lab 15, which follows up on the second example above.

#### Wed, Apr 22

• In class, we will work on:
• Introduction to decisions with hypothesis tests; errors; limitations of the framework
• Type I Errors:
• Type II Errors and Power:

#### Fri, Apr 24

• In class, we will work on:
• Warm Up about Power Functions I suggest giving the problem a try before watching the solutions video or looking at the solutions pdf.
• Warm up: pdf
• Annotated solutions: pdf
• Motivation for Neyman Pearson Lemma I suggest giving the problem a try before watching the solutions video or looking at the solutions pdf.
• Warm up: pdf
• Annotated solutions: pdf
• Set up for Neyman Pearson Lemma: reminder and more detail about likelihood ratios
• Annotated pdf: pdf
• Statement and proof of Neyman Pearson Lemma

#### Mon, Apr 27

• In class, we will work on:
• Composite likelihood ratio tests; derivation of t tests

We will have a final exam in this class.