Least Squares Theorem

$$A^T A \bar{x} = A^T b\\ \text{or}\\ \bar{x} = (A^T A)^{-1} A^T b$$

The idea is to find some vector $\bar{x}$ for a given matrix $A$ and $\vec{b}$ that satisfies $A\bar{x} = b$. But that system is in fact inconsistent; $b - A\bar{x} \neq 0$. So we look for a solution for $\bar{x}$ that comes as close as possible. The result of $b - A\bar{x} \neq 0$ will be some error $\vec{e}$. We can take the length of this error vector to express the accuracy of our solution. Optimally, we find the solution that is as close to $0$ as possible; the least squares solution.