Now that we've considered single parameter regression, let's consider linear regression. So the setting is something like this. We have some y values, and x values and they're paired. This, say, and we want to fit the line that best goes, well, that's not a very good line, the line that best goes through the data, say for example that line. And we're gonna do it, the way we're gonna define best is to take these values like this, the residuals and add them up squared. So the sum of the squared the residuals, and we're gonna try and minimize those. So see, that line is gonna have a lower sum of the squared residuals than say, for example, that line. Okay, so let's set up the problem. So, we have our y is an n by 1 vector. And our x is an n by 1 vector. And we want to minimize, norm y- beta nought times Jn- beta 1 times x norm squared. Now, you can do this by taking derivatives of course. And in this case cuz there's just two parameters, you can do it very easily. However, I wanna show how to do this using, just by referring back to regression to the origin. So let's imagine that we've fixed, and this is a technique that we can use a lot, so fix beta 1. So assume that it's known. Then these two values right there together, y- beta 1 x. Consider them a single vector, okay? So we can just write this, I know I haven't done anything. y- beta 1 x- beta nought Jn. If I think of that as a single vector, then this is just mean only regression and we know that the solution for beta nought has to be the average of this vector right here, okay? So what we know is that beta nought, as it depends on beta 1, all right, if we fix beta nought as it depends on beta 1, has to go to the point 1 over n (y- beta 1 x)Jn transpose, which works out to be y bar- beta 1 x bar. Okay, now, if we take that beta nought and plug it back into this equation, let me write, this is star, let me write star has to be greater than or equal to, if we plug that beta noughtt back in. And I'm gonna rearrange some terms a little bit. So we get y- y bar Jn + beta 1 x- x bar, times Jn. Okay, so I just did some rearranging some terms, so you can do that on your own if you don't see it immediately. So now, I can rewrite this and say, xy tilde, oops, and that's a minus sign,- beta 1 x tilde, where y tilde is the centered version of y, and x tilde is the centered version of x. Now this equation is merely regression through the origin, okay, with the new vectors y tilde and x tilde. And so we know that the solution has to be beta 1 hat equal to norm inner product of y tilde and x tilde over the inner product of x tilde by itself. And we know from the last lecture that that is the covariance, the estimated covariance between x and y times the standard deviation of the y over the standard deviation of the x. So it turns out that if we do linear regression that our estimate is identical to as if we did regression through the origin having centered the data first. Okay, now, we have to solve for the intercept. But we have the equation for the intercept right here. And so we know that as the intercept depends on beta 1, it has to take that form. So we get beta nought hat is equal to y bar minus beta 1 hat x bar. And this intercept, saying the intercept is this, is basically if I go back up to my picture right here, this basically says that if you take the average of the y's and the average of the x's that the linear regression model has to go through specifically that point. Okay so every time you fit a linear regression model, it has to go through the point y bar, x bar. Okay, well that's fitting linear regression in a nutshell.
