Singular Value Decomposition

$$M = U \Sigma V^T$$

Here, $M$ is an $m \times n$ matrix, $U$ is an $m \times m$ orthogonal matrix, $\Sigma$ is a diagonal matrix and $V$ is an $n \times n$ orthogonal matrix.

The entries in the diagonal matrix $\Sigma$ consist of the square roots of the non-zero eigenvalues of the matrix $M^T M$, ordered from greatest to least. They are called the Singular Values of M, denoted $\sigma_1, ..., \sigma_n$.

The columns of $V$ are the normalized eigenvectors belonging to the previously mentioned eigenvalues, also ordered from greatest to least. These column vectors are called the right singular vectors of M Note: $V$ should be an orthogonal matrix, so you may need to apply the Gram-Schmidt process to orthogonalize the eigenvectors.

The columns of $U$, called the left singular vectors of M, are calculated with the following formula, where $\vec{v_1}, \vec{v_n}$ are the column vectors of $V$:

$$\frac{1}{\sigma_1} M \vec{v_1}, ... \frac{1}{\sigma_n} M \vec{v_2}$$

The columns of $U$ should add up to be an orthonormal basis for $R^m$. If $m$ > $n$, you will need to apply the Gram-Schmidt process (or insight) to complete the basis.

The SVD splits the transformation $M$ into three distinct consecutive moves: a rotation ($V$), a scaling ($\Sigma$) and a final rotation ($U$).