Covariance

Description:

• To find the “correlation” between 2 variables, and Correlation is the standard value to easier understood
• If $X$ and $Y$ are independent, then their covariance, $\text{Cov}(X,Y)=0$
• But $\text{Cov}(X,Y)=0$ doesn’t mean that they are independent
• $\text{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])]=E[XY]-E[X]E[Y]$
  • Expected value

Propositions:

• $\text{Cov}(X,Y)=\text{Cov}(Y,X)$
• $\text{Cov}(X,X)=Var(X)$
  • Variance
• $\text{Cov}(aX,Y)=a\text{Cov}(X,Y)$
• $\text{Cov}(-X,Y)=-\text{Cov}(X,Y)$
• $\text{Cov}(\sum _{i=1}^n{X}_i,\sum _{j=1}^m{Y}_j)=\sum _{i=1}^n\sum _{j=1}^m\text{Cov}(X_i,Y_j)$
• If $Y_j=X_j,j=1,...,n$
  • $Var\left(\sum _{i=1}^n{X}_i\right)=\sum _{i=1}^{n}Var(X_i)+\sum _{i\ne j}\text{Cov}(X_i,X_j)$ $=\sum _{i=1}^{n}Var(X_i)+2\sum _{i<j}\text{Cov}(X_i,X_j)$
  • If $X_i,...,X_n$ are pairwise independent, then $Var\left(\sum _{i=1}^n{X}_i\right)=\sum _{i=1}^{n}Var(X_i)$

Covariance matrix:

• Symmetric Matrix and PSD with each entry ${\Sigma }_{ij}$ is the variance/covariance of $i$ and $j$
• Given $m$ points $x^{(1)},...,x^{(m)}$ in Rn, we define the sample covariance matrix to be the $n\times n$ symmetric matrix $C\doteq \frac{1}{m}{\sum }_{i=1}^m(x^{(i)}-\hat{x})(x^{(i)}-\hat{x}{)}^{⊺}$ where:
  • $\hat{x}\in {\mathbb{R}}^n$ is the sample average of the points
• Arises when computing the sample variance of the scalar products $s_i\doteq w^{⊺}{x}^{(i)},i=1,...,m$
  • where $w\in {\mathbb{R}}^n$ is a given vector: denoting by $\hat{s}$ the average of the values $s_1,...,s_m$
  • we have ${\sigma }^2=\frac{1}{m}\sum _{i=1}^m(w^{⊺}{x}^{(i)}-\hat{s}{)}^2=\frac{1}{m}\sum _{i=1}^m(w^{⊺}(x^{(i)}-\hat{x}){)}^2=w^{⊺}Cw$

Applications:

• Portfolio variance:
  • For $n$ financial assets, we can define a vector $r\in {\mathbb{R}}^n$ whose components $r_k$ are the rate of returns of the $k$-th asset, $k=1,\dots ,n$.
  • Assume now that we have observed $m$ samples of historical returns $r^{(t)},t=1,\dots ,m$. The sample average over that history of return is $\hat{r}=(1/m)\left(r^{(1)}+\dots +r^{(m)}\right)$, and the sample covariance matrix has $(i,j)$ component given by:
    • $C_{ij}=\frac{1}{m}{\sum }_{t=1}^m\left(r_i^{(t)}-{\hat{r}}_i\right)\left(r_j^{(t)}-{\hat{r}}_j\right),1\le i,j\le n.$
  • If $w\in {\mathbb{R}}^n$ represents a portfolio “mix,” that is $w_k\ge 0$ is the fraction of the total wealth invested in asset $k$, then the return of such a portfolio is given by $\rho =r^{T}w$.
  • The sample average of the portfolio return is ${\hat{r}}^{T}w$, while the sample variance is given by $w^T{C}_w$
• Hessian Matrix
  • The Hessian of a twice differentiable function $f:{\mathbb{R}}^n\to \mathbb{R}$ at a point $x\in \mathrm{dom}f$ is the matrix containing the second derivatives of the function at that point. That is, the Hessian is the matrix with elements given by
    • $H_{ij}=\frac{{\partial }^{2}f(x)}{\partial x_i\partial x_j},1\le i,j\le n$
  • The Hessian of $f$ at $x$ is often denoted as ${\nabla }^{2}f(x)$.
  • Since the second-derivative is independent of the order in which derivatives are taken, it follows that $H_{ij}=H_{jj}$ for every pair $(i,j)$, thus the Hessian is always a symmetric matrix.