Applied linear operators and spectral methods/Lecture 4

More on spectral decompositions

In the course of the previous lecture we essentially proved the following theorem:

Theorem:

1) If a $n\times n$ matrix $𝐀$ has $n$ linearly independent real or complex eigenvectors, the $𝐀$ can be diagonalized. 2) If $𝐓$ is a matrix whose columns are eigenvectors then $𝐓𝐀{𝐓}^{-1}=\mathrm{\Lambda }$ is the diagonal matrix of eigenvalues.

The factorization $𝐀={𝐓}^{-1}\mathrm{\Lambda }𝐓$ is called the spectral representation of $𝐀$.

Application

We can use the spectral representation to solve a system of linear homogeneous ordinary differential equations.

For example, we could wish to solve the system

$\frac{d𝐮}{dt}=𝐀𝐮=\left[\begin{array}{cc}-2& 1\\ 1& -2\end{array}\right]\left[\begin{array}{c}{u}_1\\ u_2\end{array}\right]$

(More generally $𝐀$ could be a $n\times n$ matrix.)

Comment:

Higher order ordinary differential equations can be reduced to this form. For example,

$\frac{d^2{u}_1}{d{t}^2}+a\frac{d{u}_1}{dt}=b{u}_1$

Introduce

$u_2=\frac{d{u}_1}{dt}$

Then the system of equations is

$\begin{array}{cc}\frac{d{u}_1}{dt}& =u_2\\ \frac{d{u}_2}{dt}& =b{u}_1-a{u}_2\end{array}$

or,

$\frac{d𝐮}{dt}=\left[\begin{array}{cc}0& 1\\ b& -a\end{array}\right]\left[\begin{array}{c}{u}_1\\ u_2\end{array}\right]=𝐀𝐮$

Returning to the original problem, let us find the eigenvalues and eigenvectors of $𝐀$. The characteristic equation is

$det(𝐀-\lambda 𝐈)=0$

o we can calculate the eigenvalues as

$(2+\lambda )(2+\lambda )-1=0⟹{\lambda }^2+4\lambda +3=0⟹{\lambda }_1=-1,{\lambda }_2=-3$

The eigenvectors are given by

$(𝐀-{\lambda }_1𝐈){𝐧}_1=\mathrm{𝟎};(𝐀-{\lambda }_2𝐈){𝐧}_2=\mathrm{𝟎}$

or,

$-n_1^1+n_2^1=0;n_1^1-n_2^1=0;n_1^2+n_2^2=0;n_1^2+n_2^2=0$

Possible choices of ${𝐧}_1$ and ${𝐧}_2$ are

${𝐧}_1=\left[\begin{array}{c}1\\ 1\end{array}\right];{𝐧}_2=\left[\begin{array}{c}1\\ -1\end{array}\right]$

The matrix $𝐓$ is one whose columd are the eigenvectors of $𝐀$, i.e.,

$𝐓=\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]$

and

$\mathrm{\Lambda }={𝐓}^{-1}𝐀𝐓=\left[\begin{array}{cc}-1& 0\\ 0& -3\end{array}\right]$

If $𝐮=𝐓{𝐮}^{\text{'}}$ the system of equations becomes

$\frac{d{𝐮}^{\text{'}}}{dt}={𝐓}^{-1}𝐀𝐓{𝐮}^{\text{'}}=\mathrm{\Lambda }{𝐮}^{\text{'}}$

Expanded out

$\frac{d{u}_1^{\text{'}}}{dt}=-u_1^{\text{'}};\frac{d{u}_2^{\text{'}}}{dt}=-3u_2^{\text{'}}$

The solutions of these equations are

$u_1^{\text{'}}=C_1{e}^{-t};u_2^{\text{'}}=C_2{e}^{-3t}$

Therefore,

$𝐮=𝐓{𝐮}^{\text{'}}=\left[\begin{array}{c}{C}_1{e}^{-t}+C_2{e}^{-3t}\\ C_1{e}^{-t}-C_2{e}^{-3t}\end{array}\right]$

This is the solution of the system of ODEs that we seek.

Most "generic" matrices have linearly independent eigenvectors. Generally a matrix will have $n$ distinct eigenvalues unless there are symmetries that lead to repeated values.

Theorem

If $𝐀$ has $k$ distinct eigenvalues then it has $k$ linearly independent eigenvectors.

Proof:

We prove this by induction.

Let ${𝐧}_j$ be the eigenvector corresponding to the eigenvalue ${\lambda }_j$. Suppose ${𝐧}_1,{𝐧}_2,\dots ,{𝐧}_{k-1}$ are linearly independent (note that this is true for $k$ = 2). The question then becomes: Do there exist ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_k$ not all zero such that the linear combination

${\alpha }_1{𝐧}_1+{\alpha }_2{𝐧}_2+\dots +{\alpha }_k{𝐧}_k=0$

Let us multiply the above by $(𝐀-{\lambda }_k𝐈)$. Then, since $𝐀{𝐧}_i={\lambda }_i{𝐧}_i$, we have

${\alpha }_1({\lambda }_1-{\lambda }_k){𝐧}_1+{\alpha }_2({\lambda }_2-{\lambda }_k){𝐧}_2+\dots +{\alpha }_{k-1}({\lambda }_{k-1}-{\lambda }_k){𝐧}_{k-1}+{\alpha }_k({\lambda }_k-{\lambda }_k){𝐧}_k=\mathrm{𝟎}$

Since ${\lambda }_k$ is arbitrary, the above is true only when

${\alpha }_1={\alpha }_2=\dots ={\alpha }_{k-1}=0$

In thast case we must have

${\alpha }_k{𝐧}_k=\mathrm{𝟎}⟹{\alpha }_k=0$

This leads to a contradiction.

Therefore ${𝐧}_1,{𝐧}_2,\dots ,{𝐧}_k$ are linearly independent. $◻$

Another important class of matrices which are diagonalizable are those which are self-adjoint.

Theorem

If $𝑨$ is self-adjoint the following statements are true

1. $⟨𝑨𝐱,𝐱⟩$ is real for all $𝐱$.
2. All eigenvalues are real.
3. Eigenvectors of distinct eigenvalues are orthogonal.
4. There is an orthonormal basis formed by the eigenvectors.
5. The matrix $𝑨$ can be diagonalized (this is a consequence of the previous statement.)

Proof

1) Because the matrix is self-adjoint we have

$⟨𝑨𝐱,𝐱⟩=⟨𝐱,𝑨𝐱⟩$

From the property of the inner product we have

$⟨𝐱,𝑨𝐱⟩=\overline{⟨𝑨𝐱,𝐱⟩}$

Therefore,

$⟨𝑨𝐱,𝐱⟩=\overline{⟨𝑨𝐱,𝐱⟩}$

which implies that $⟨𝑨𝐱,𝐱⟩$ is real.

2) Since $⟨𝑨𝐱,𝐱⟩$ is real, $⟨𝑰𝐱,𝐱⟩=⟨𝐱,𝐱⟩$ is real. Also, from the eiegnevalue problem, we have

$⟨𝑨𝐱,𝐱⟩=\lambda ⟨𝐱,𝐱⟩$

Therefore, $\lambda$ is real.

3) If $(\lambda ,𝐱)$ and $(\mu ,𝐲)$ are two eigenpairs then

$\lambda ⟨𝐱,𝐲⟩=⟨𝑨𝐱,𝐲⟩$

Since the matrix is self-adjoint, we have

$\lambda ⟨𝐱,𝐲⟩=⟨𝐱,𝑨𝐲⟩=\mu ⟨𝐱,𝐲⟩$

Therefore, if $\lambda \ne \mu \ne 0$, we must have

$⟨𝐱,𝐲⟩=0$

Hence the eigenvectors are orthogonal.

4) This part is a bit more involved. We need to define a manifold first.

Linear manifold

A linear manifold (or vector subspace) $ℳ\in 𝒮$ is a subset of $𝒮$ which is closed under scalar multiplication and vector addition.

Examples are a line through the origin of $n$-dimensional space, a plane through the origin, the whole space, the zero vector, etc.

Invariant manifold

An invariant manifold $ℳ$ for the matrix $𝑨$ is the linear manifold for which $𝐱\in ℳ$ implies $𝑨𝐱\in ℳ$.

Examples are the null space and range of a matrix $𝑨$. For the case of a rotation about an axis through the origin in $n$-space, invaraiant manifolds are the origin, the plane perpendicular to the axis, the whole space, and the axis itself.

Therefore, if ${𝐱}_1,{𝐱}_2,\dots ,{𝐱}_m$ are a basis for $ℳ$ and ${𝐱}_{m+1},\dots ,{𝐱}_n$ are a basis for ${ℳ}_{\perp }$ (the perpendicular component of $ℳ$) then in this basis $𝑨$ has the representation

$𝑨=\left[\begin{array}{ccccc}x& x& |& x& x\\ x& x& |& x& x\\ -& -& -& -& -\\ 0& 0& |& x& x\\ 0& 0& |& x& x\end{array}\right]$

We need a matrix of this form for it to be in an invariant manifold for $𝑨$.

Note that if $ℳ$ is an invariant manifold of $𝑨$ it does not follow that ${ℳ}_{\perp }$ is also an invariant manifold.

Now, if $𝑨$ is self adjoint then the entries in the off-diagonal spots must be zero too. In that case, $𝑨$ is block diagonal in this basis.

Getting back to part (4), we know that there exists at least one eigenpair (${\lambda }_1,{𝐱}_1$) (this is true for any matrix). We now use induction. Suppose that we have found ($n-1$) mutually orthogonal eigenvectors ${𝐱}_i$ with $𝑨{𝐱}_i={\lambda }_i{𝐱}_i$ and ${\lambda }_i$ are real, $i=1,\dots ,k-1$. Note that the ${𝐱}_i$s are invariant manifolds of $𝑨$ as is the space spanned by the ${𝐱}_i$s and so is the manifold perpendicular to these vectors).

We form the linear manifold

${ℳ}_k=\{𝐱|⟨𝐱,{𝐱}_j⟩=0j=1,2,\dots ,k-1\}$

This is the orthogonal component of the $k-1$ eigenvectors ${𝐱}_1,{𝐱}_2,\dots ,{𝐱}_{k-1}$ If $𝐱\in {ℳ}_k$ then

$⟨𝐱,{𝐱}_j⟩=0\text{and}⟨𝑨𝐱,{𝐱}_j⟩=⟨𝐱,𝑨{𝐱}_j⟩={\lambda }_j⟨𝐱,{𝐱}_j⟩=0$

Therefore $𝑨𝐱\in {ℳ}_k$ which means that ${ℳ}_k$ is invariant.

Hence ${ℳ}_k$ contains at least one eigenvector ${𝐱}_k$ with real eigenvalue ${\lambda }_k$. We can repeat the procedure to get a diagonal matrix in the lower block of the block diagonal representation of $𝑨$. We then get $n$ distinct eigenvectors and so $𝑨$ can be diagonalized. This implies that the eigenvectors form an orthonormal basis.

5) This follows from the previous result because each eigenvector can be normalized so that $⟨{𝐱}_i,{𝐱}_j⟩={\delta }_{ij}$.

We will explore some more of these ideas in the next lecture. Template:Lecture