---
title: "Problem Set 5: Written Part"
author: "Your Name Goes Here"
output:
pdf_document:
keep_tex: true
header-includes:
- \usepackage{booktabs}
geometry: margin=1.5cm
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(tidyverse)
```
\def\simiid{\stackrel{{\mbox{\text{\tiny i.i.d.}}}}{\sim}}
# Details
### How to Write Up
The written part of this assignment can be either typeset using latex or hand written.
### Grading
5% of your grade on this assignment is for turning in something legible. This means it should be organized, and any Rmd files should knit to pdf without issue.
An additional 15% of your grade is for completion. A quick pass will be made to ensure that you've made a reasonable attempt at all problems.
Across both the written part and the R part, in the range of 1 to 3 problems will be graded more carefully for correctness. In grading these problems, an emphasis will be placed on full explanations of your thought process. You don't need to write more than a few sentences for any given problem, but you should write complete sentences! Understanding and explaining the reasons behind what you are doing is at least as important as solving the problems correctly.
Solutions to all problems will be provided.
### Collaboration
You are allowed to work with others on this assignment, but you must complete and submit your own write up. You should not copy large blocks of code or written text from another student.
### Sources
You may refer to our text, Wikipedia, and other online sources. All sources you refer to must be cited.
\newpage
# Problem I: Binomial Distribution
We have previously found that if $X \sim \text{Binomial}(n, \theta)$ then the maximum likelihood estimator of $\theta$ is $\hat{\theta}^{MLE} = \frac{X}{n}$. We showed that $E(\hat{\theta}^{MLE}) = \theta$, $Var(\hat{\theta}^{MLE}) = \frac{\theta(1-\theta)}{n}$, and $MSE(\hat{\theta}^{MLE}) = \frac{\theta(1-\theta)}{n}$.
#### (a) Is it possible for another estimator $\tilde{\Theta}$ to have lower variance than the MSE? If so, give an example of such an estimator. If not, explain why not with reference to a theorem from class.
\vspace{7cm}
#### (b) Is it possible for another *unbiased* estimator $\tilde{\Theta}$ to have lower variance than the MSE? If so, give an example of such an estimator. If not, explain why not with reference to a theorem from class.
\newpage
# Problem II: Exponential Distribution
Suppose $X_1, \ldots, X_n \sim \text{Exp}(\lambda)$, where the $X_i$ are independent.
Here are some facts about the exponential distribution (please use this parameterization of the exponential distribution for this problem):
If $X \sim Exp(\lambda)$ then the density function is given by $f(x | \lambda) = \lambda e^{-\lambda x}$
The mean and variance are given by $E(X) = \frac{1}{\lambda}$ and $Var(X) = \frac{1}{\lambda^2}$.
Based on a sample $x_1, \ldots, x_n$, the log-likelihood function is:
\begin{align*}
L(\lambda | x_1, \ldots, x_n) = n \log(\lambda) - \lambda \sum_{i = 1}^n x_i
\end{align*}
The first derivative of the log-likelihood function is:
\begin{align*}
\frac{d}{d \lambda} L(\lambda | x_1, \ldots, x_n) = \frac{n}{\lambda} - \sum_{i = 1}^n x_i
\end{align*}
The second derivative of the log-likelihood function is:
\begin{align*}
\frac{d^2}{d \lambda^2} L(\lambda | x_1, \ldots, x_n) = - \frac{n}{\lambda^2}
\end{align*}
The maximum likelihood estimator can be shown to be: $\hat{\lambda}_{MLE} = 1 / \bar{X}$.
#### (a) Find a normal approximation to the sampling distribution of the maximum likelihood estimator. This will depend on the unknown parameter $\lambda$.
\newpage
#### (b) Find a normal approximation to the posterior distribution of $\Lambda$ in a Bayesian analysis. Base your approximation on the maximum likelihood estimate $\hat{\lambda}_{MLE}$.