Orthogonality

Description:

• 2 n-vectors are orthogonal if $x^{T}y=0\to x\perp y$
• Non zero vectors are said to be mutually orthogonal if each vector is orthogonal to all other vectors
  • they are also Linear Independence

Orthonormal:

• A collections of vectors $S=\{x^{(1)},...,x^{(d)}\}$ is said to be orthonormal if for $i,j=1,...d$
  • $(x^{(i)}{)}^T{x}^{(j)}=\{\begin{array}{ll}0& \text{if }i\ne j\\ 1& \text{if }i=j\end{array}$, meaning they are all unit vectors
• In words, $S$ is orthonormal if every element has unit norm, and all elements are orthogonal to eachother
• A collection of orthonormal vectors $S$ form an orthonomal basis for the span of $S$

Orthogonalization:

• Procedure that finds an orthonormal basis of the span of given vectors.
• Given vetors $a_1,...,a_k\in {\mathbb{R}}^n$, an orthogonalization procedure computes vectors $q_1,...,q_n\in {\mathbb{R}}^n$ such that $S:=span\{a_1,...,a_k\}=span\{q_1,...,q_r\}$
  • where $r$ is the dimension of $S$ and $q_i^T{q}_j=1(i\ne j)$ for $1\le i,j\le r$
  • and $q_i^T{q}_i=1$ for $1\le i,j\le r$
  • meaning all the $q$ vectors form an orthonormal basis for the span
• Gram-Schmidt Algorithm