## Simple Linear Regression with distinct x values – Everything Is OK

Suppose we have 3 observations:

• $$y_1 = 2$$, $$x_1 = 1$$ (first observation is in the first group)
• $$y_2 = 4$$, $$x_2 = 2$$ (second observation is in the second group)
• $$y_3 = 5$$, $$x_3 = 3$$ (third observation is in the second group)

Our model is

$y_i = \beta_0 + \beta_1 x_i + \varepsilon_i \\ \varepsilon_i \sim \text{Normal}(0, \sigma^2)$

The design matrix is $$X = \begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix}$$

We know that we can use the following code to find the estimates of $$\beta_0$$ and $$\beta_1$$:

X <- cbind(
c(1, 1, 1),
c(1, 2, 3)
)
y <- matrix(c(2, 4, 5))

beta_hat <- solve(t(X) %*% X) %*% t(X) %*% y
beta_hat
##           [,1]
## [1,] 0.6666667
## [2,] 1.5000000

Here is a picture of the RSS as a function of $$\beta_0$$ and $$\beta_1$$, with our estimates $$(\hat{\beta}_0, \hat{\beta}_1)$$ shown with a red point:

$RSS = \sum_{i = 1}^n (y_i - \hat{y}_i)^2 = \{2 - (\beta_0 + \beta_1 \cdot 1)\}^2 + \{4 - (\beta_0 + \beta_1 \cdot 2)\}^2 + \{5 - (\beta_0 + \beta_1 \cdot 3)\}^2$