Summary

Orthonormal Basis

Definition

Definition

An orthonormal basis for a vector space $V$ is a basis $\{v_1,v_2,\dots ,v_n\}$ such that $⟨v_i,v_j⟩=\{\begin{array}{ll}1& \text{if }i=j\\ 0& \text{if }i\ne j\end{array}$ where $⟨\cdot ,\cdot ⟩$ is the inner product on $V$.

Linearly Dependent in Orthonormal Basis

If $V$ is a vector space with an orthonormal basis $\{v_1,v_2,\dots ,v_n\}$, then it is linearly independent

Properties

Theorem

Let $V$ be a vector space with an orthonormal basis $\{v_1,v_2,\dots ,v_n\}$. Then, for any vector $v\in V$, the vector $v$ can be expressed as $v={\sum }_{i=1}^n⟨v,v_i⟩v_i$

Projections

Definition

Let $V$ be a vector space and the $H$ be a subspace of $V$ with an orthonormal basis $\{v_1,v_2,\dots ,v_n\}$. The projection of a vector $v\in V$ onto $H$ is given by ${\text{proj}}_H(v)={\sum }_{i=1}^n⟨v,v_i⟩v_i$

Orthogonal Complementb

Definition

Let $V$ be a vector space and $W$ be a subspace of $V$. The orthogonal complement of $W$ is the set of all vectors in $V$ that are orthogonal to every vector in $W$. It is denoted by $W^{\perp }$. $W^{\perp }=\{v\in V:⟨v,w⟩=0,\forall w\in W\}$

Example How to Find Orthogonal Complement

Let $V={\mathbb{R}}^3$ and $W=\text{span}\{(1,1,0),(1,0,1)\}$. The orthogonal complement of $W$ is given by $W^{\perp }=\{v\in {\mathbb{R}}^3:⟨v,w⟩=0,\forall w\in W\}$.

1. Let $v=(x,y,z)\in W^{\perp }$ and $w_1=(1,1,0)$, $w_2=(1,0,1)\in W$. Then, $⟨v,w_1⟩=0$ and $⟨v,w_2⟩=0$. This gives the system of equations $x+y=0$ and $x+z=0$.
2. The solution to the system is $V=\text{span}\{(1,0,0),(0,1,-1)\}$.
3. Therefore, $W^{\perp }=\text{span}\{(1,0,0),(0,1,-1)\}$.

Theorem

Let $V$ be a vector space and $W\subset V$ be a subspace with an orthonormal basis $\{v_1,v_2,\dots ,v_n\}$. Then,

1. $W^{\perp }\subset V$
2. $W\cap W^{\perp }=\{\vec{0}\}$
3. $(W^{\perp }{)}^{\perp }=W$
4. $\dim (W)+\dim (W^{\perp })=\dim (V)$

Theorem

Let $V$ be a vector space. We can write $V=W\oplus W^{\perp }$ or $v={\text{proj}}_W(v)+{\text{proj}}_{W^{\perp }}(v)$ for any vector $v\in V$

Orthogonal Completment of Nullspace

$\text{ker}(A{)}^{\perp }=R_A$ and $\text{ker}(A^T)=C_A$

Gramm-Schmidt Orthogonalization

Definition

The Gramm-Schmidt orthogonalization process takes a set of linearly independent vectors $\{v_1,v_2,\dots ,v_n\}$ and produces an orthonormal basis $\{u_1,u_2,\dots ,u_n\}$ for the subspace spanned by the original vectors.

Procedure

1. Set $u_1=\frac{v_1}{\Vert v_1\Vert }$
2. For $i=2,3,\dots ,n$
   1. set $u_i=v_i-{\sum }_{j=1}^{i-1}⟨v_i,u_j⟩u_j$
   2. set $u_i=\frac{u_i}{\Vert u_i\Vert }$

Example

Consider the vectors $v_1=(1,1,0)$, $v_2=(1,0,1)$, and $v_3=(0,1,1)$ in ${\mathbb{R}}^3$. The Gramm-Schmidt orthogonalization process produces the orthonormal basis $\{u_1,u_2,u_3\}$ as follows:

1. Set $u_1=\frac{v_1}{\Vert v_1\Vert }=\frac{(1,1,0)}{\sqrt{2}}=\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}},0\right)$
2. Set $u_2=v_2-⟨v_2,u_1⟩u_1=(1,0,1)-\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}},0\right)=\left(\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}},1\right)$
3. Set $u_2=\frac{u_2}{\Vert u_2\Vert }=\left(\frac{1}{2\sqrt{2}},-\frac{1}{2\sqrt{2}},\frac{1}{2}\right)$
4. Set $u_3=v_3-⟨v_3,u_1⟩u_1-⟨v_3,u_2⟩u_2=(0,1,1)-\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}},0\right)-\left(\frac{1}{2\sqrt{2}},-\frac{1}{2\sqrt{2}},\frac{1}{2}\right)=\left(-\frac{1}{2\sqrt{2}},\frac{1}{2\sqrt{2}},\frac{1}{2}\right)$
5. Set $u_3=\frac{u_3}{\Vert u_3\Vert }=\left(-\frac{1}{2},\frac{1}{2},\frac{1}{2}\right)$

The orthonormal basis is $\left\{\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}},0\right),\left(\frac{1}{2\sqrt{2}},-\frac{1}{2\sqrt{2}},\frac{1}{2}\right),\left(-\frac{1}{2},\frac{1}{2},\frac{1}{2}\right)\right\}$