# Singular Value Decomposition

$$
M = U \Sigma V^T
$$

Here, $$M$$ is an $$m \times n$$ matrix, $$U$$ is an $$m \times m$$ orthogonal matrix, $$\Sigma$$ is a diagonal matrix and $$V$$ is an $$n \times n$$ orthogonal matrix.

The entries in the diagonal matrix $$\Sigma$$ consist of the square roots of the non-zero eigenvalues of the matrix $$M^T M$$, ordered from greatest to least. They are called the *Singular Values* of M, denoted $$\sigma\_1, ..., \sigma\_n$$.

The columns of $$V$$ are the normalized eigenvectors belonging to the previously mentioned eigenvalues, also ordered from greatest to least. These column vectors are called the *right singular vectors of M* Note: $$V$$ should be an orthogonal matrix, so you may need to apply the *Gram-Schmidt process* to orthogonalize the eigenvectors.

The columns of $$U$$, called the *left singular vectors of M*, are calculated with the following formula, where $$\vec{v\_1}, \vec{v\_n}$$ are the column vectors of $$V$$:

$$
\frac{1}{\sigma\_1} M \vec{v\_1}, ... \frac{1}{\sigma\_n} M \vec{v\_2}
$$

The columns of $$U$$ should add up to be an orthonormal basis for $$R^m$$. If $$m$$ > $$n$$, you will need to apply the Gram-Schmidt process (or insight) to complete the basis.

The SVD splits the transformation $$M$$ into three distinct consecutive moves: a rotation ($$V$$), a scaling ($$\Sigma$$) and a final rotation ($$U$$).

![](https://791392235-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LadqTMEugj8cHTeiI0A%2F-LajRaVUrGax6lB38Mz-%2F-LajRb1weV-YgMh1Cf5h%2Fsvd.gif?generation=1553426707612992\&alt=media)