Parameter Calculation by Least Squares Minimization

The method of least squares consists of minimizing with respect to b. Setting q = Xb, we minimize:

subject to:

Let be the least squares estimate of b. The fitted regression is denoted by:

The elements of are called the residuals. The value of:

 

is called the residual sum of squares. The matrix:

which is the regression matrix without the first column of 1s, is called the predictor data matrix.