Set Up
Suppose we have 3 observations:
- \(y_1 = 1\), \(x_1 = 0\) (first observation is in the first group)
- \(y_2 = 2\), \(x_2 = 1\) (second observation is in the second group)
- \(y_3 = 3\), \(x_3 = 1\) (third observation is in the second group)
Our model is
\[
y_i = \beta_0 + \beta_1 x_i + \varepsilon_i \\
\varepsilon_i \sim \text{Normal}(0, \sigma^2)
\]
The design matrix is \(X = \begin{bmatrix} 1 & 0 \\ 1 & 1 \\ 1 & 1 \end{bmatrix}\)
The fitted values are \(\hat{y} = \begin{bmatrix} 1 \\ 2.5 \\ 2.5 \end{bmatrix}\)
Illustration of fitted values
In the graphic below,
- The green vectors show the columns of \(X\); the green plane is the column space of \(X\).
- The blue point is the observed vector \(y\)
- The red point is the fitted vector \(\hat{y}\); the red line shows the projection of \(y\) into the column space of \(X\).