Symmetric Matrix

Description:

• A square matrix $A\in {\mathbb{R}}^{n,n}$ is symmetric if it is equal to its transpose: $A=A^{⊺}$
• The set of symmetric $n\times n$ matrices is is a subspace of ${\mathbb{R}}^{n,n}$, and it is denoted with $S^n$

Positive Semi-definite Matrix

Spectral Theorem

Variational characterization of eigenvalues

• Since the eigenvalues of $A\in {\mathbb{S}}^n$ are real, we can arrange them in decreasing order: ${\lambda }_{\max }(A)={\lambda }_1(A)\ge {\lambda }_2(A)\ge \cdots \ge {\lambda }_n(A)={\lambda }_{\min }(A)$
• The extreme eigenvalues can be related to the minimum and the maximum attained by the quadratic form induced by $A$ over the unit Euclidean sphere.
• For $x\ne 0$ the ratio $\frac{x^{\top }Ax}{x^{\top }x}$ is called a Rayleigh quotient

Rayleigh quotient:

• theorem Given $A\in {\mathbb{S}}^n$, it holds that ${\lambda }_{\min }(A)\le \frac{x^{T}Ax}{x^{T}x}\le {\lambda }_{\max }(A),\forall x\ne 0$
• Moreover, $\begin{array}{rl}& {\lambda }_{\max }(A)=\underset{x:\Vert x{\Vert }_2=1}{\max }{x}^{T}Ax\\ & {\lambda }_{\min }(A)=\underset{x:\Vert x{\Vert }_2=1}{\min }{x}^{T}Ax,\end{array}$
  • and the maximum and minimum are attained for $x=u_1$ and for $x=u_n$, respectively
  • where $u_1$ (resp. $u_n$ ) is the unit-norm eigenvector of $A$ associated with its largest (resp. smallest) eigenvalue of $A$.