Syllabus Map

• Study map: Syllabus Study Map

Core Idea

• Orthogonal vectors have zero dot product.
• Projection decomposes a vector into explained and residual components.

Key Formula

Projection of $x$ onto $u$:

Practical Notes

Residuals are orthogonal to fitted subspace in least squares.

• This gives geometric intuition for regression fit quality.

Orthonormal bases simplify computations.

• Dot products and decompositions become more stable.

Why This Matters for ML

• Linear regression is projection of targets onto the feature span.
• Residual orthogonality provides a geometric check of least-squares solutions.
• Orthogonal bases reduce numerical coupling and simplify optimization.
• Subspace projection ideas underlie PCA and many representation methods.