Least Squares Theorem

ATAxˉ=ATborxˉ=(ATA)1ATbA^T A \bar{x} = A^T b\\ \text{or}\\ \bar{x} = (A^T A)^{-1} A^T b

The idea is to find some vector xˉ\bar{x} for a given matrix AA and b\vec{b} that satisfies Axˉ=bA\bar{x} = b. But that system is in fact inconsistent; bAxˉ0b - A\bar{x} \neq 0. So we look for a solution for xˉ\bar{x} that comes as close as possible. The result of bAxˉ0b - A\bar{x} \neq 0 will be some error e\vec{e}. We can take the length of this error vector to express the accuracy of our solution. Optimally, we find the solution that is as close to 00 as possible; the least squares solution.

Last updated