Syllabus Map

• Study map: Syllabus Study Map

Core Definition

For square matrix $A$, an eigenvector $v \ne 0$ and eigenvalue $\lambda$ satisfy:

• $A$ acts on $v$ by scaling (and possibly sign flip), not changing its direction.

How to Compute Eigenvalues

Step 1: Characteristic Equation

Solve:

This polynomial is the characteristic polynomial.

Step 2: Find Eigenvectors

For each eigenvalue $\lambda$, solve:

to get non-zero vectors in that null space.

2x2 Example

Let

Characteristic equation:

So eigenvalues are $\lambda_1=3$, $\lambda_2=1$.

Corresponding eigenvectors (up to scaling):

• For $\lambda=3$: $v_1=\begin{bmatrix}1\\1\end{bmatrix}$
• For $\lambda=1$: $v_2=\begin{bmatrix}1\\-1\end{bmatrix}$

Geometric Intuition

• Most vectors are rotated/sheared/scaled by $A$.
• Eigenvectors are special directions that remain on the same line.
• Eigenvalues tell how much stretching/compression occurs along those directions.

Diagonalisation

If $A$ has enough independent eigenvectors, then:

where:

• Columns of $P$ are eigenvectors
• $D$ is diagonal with eigenvalues

Then powers are easy:

Symmetric Matrices (Very Important)

If $A=A^\top$:

• All eigenvalues are real
• Eigenvectors for distinct eigenvalues are orthogonal
• Decomposition becomes:

with orthonormal $Q$.

This is the basis of PCA and many optimisation results.

Connection to SVD

For any matrix $X$:

• Columns of $V$ are eigenvectors of $X^\top X$
• Columns of $U$ are eigenvectors of $XX^\top$
• Singular values satisfy:

Why This Matters for ML

• PCA, spectral clustering, and graph-based methods depend on eigenvectors/eigenvalues.
• Eigenvalues indicate variance/energy concentration and stability properties of transformations.
• Iterative optimization and dynamical behavior are often explained by spectral radius and modes.
• Symmetric eigendecompositions are foundational in covariance analysis and many kernels.