Orthogonality

Orthogonal set and basis

An orthogonal set is a set in which all pairs of distinct vectors are orthogonal. Similarly, an orthogonal basis is a basis that is an orthogonal set.

If $\{\vec{v_1}, ..., \vec{v_n}\}$ is an orthogonal basis for a subspace $W$ and $\vec{w}$ is a vector in $W$, the scalars $c_i, ..., c_n$ that express $w$ as a linear combination of $\{\vec{v_1}, ..., \vec{v_n}\}$ are given by:

$$c_i = \frac{w \cdot v_i}{v_i \cdot v_i} \text{for } i = 1, ..., n$$

Orthonormal set and basis

A set of vectors in $R^n$ is called an orthonormal set if it is an orthogonal set of unit vectors. Likewise, an orthonormal basis is an orthonormal set that is a basis for some subspace. You can easily obtain an orthonormal set from an orthogonal set by normalizing each vector in the set.

If $\{\vec{q_1}, \vec{q_2}, ..., \vec{q_n}\}$ is an orthonormal basis for subspace $W$ and $\vec{w}$ is in $W$, then $\vec{w}$ can be expressed as a combination of projections:

$$w = (w \cdot q_1)q_1 + (w \cdot q_2)q_2 + ... + (w \cdot q_n)q_n$$

Orthogonal Matrix

Somewhat confusingly, a square matrix in which the columns form an orthonormal set is called an orthogonal matrix. There is no name for a non-square matrix with orthonormal columns.

An orthogonal matrix some nice properties. Some of these are a bit obvious but it's nice to have them in a list. For any orthogonal matrix $Q$ with dimension $n \times n$:

• $Q^{-1} = Q^T$.

• $||Q\vec{x}|| = ||\vec{x}||$ for any $x$ in $R^n$.

• Any eigenvalues $\lambda$ of $Q$ have $|\lambda| = 1$.

• $Q \vec{x} \cdot Q \vec{y} = \vec{x} \cdot \vec{y}$ for every $x$ and $y$ in $R^n$.

• $Q^{-1}$ is orthogonal.

• $det(Q) = \pm 1$.

Orthogonal Projection

If $W$ is some subspace of $R^n$ and $\{\vec{u_1}, ..., \vec{u_n}\}$ is an orthogonal basis for $W$, for any $\vec{v}$ in $R^n$, the orthogonal projection of v onto W is defined as:

$$proj_W (v) = (\frac{u_1 \cdot v}{u_1 \cdot u_1})u_1 + ... + (\frac{u_n \cdot v}{u_n \cdot u_n})u_n$$

The component of v orthogonal to W is the vector:

$$perp_W (v) = v - proj_W (v)$$