SVD (Singular Value Decomposition) Demo

Explore Singular Value Decomposition with visualizations and dimensionality reduction examples

Overview

The SVD Demo explores Singular Value Decomposition, one of the most powerful matrix factorizations in linear algebra. SVD decomposes any matrix into three components (A = UΣV^T): two rotation matrices and one scaling matrix, revealing that any linear transformation is fundamentally just a rotation, followed by scaling, followed by another rotation.

Tips

  1. Examine the Singular Values: The singular values in Σ are ordered from largest to smallest and indicate the “importance” of each direction - rapid decay suggests the matrix has low effective rank and can be well-approximated with fewer dimensions.

  2. Try Low-Rank Approximation: Experiment with keeping only the top k singular values to see how much information you can retain with fewer dimensions - this is the foundation of data compression and dimensionality reduction.

  3. Start with Small Matrices: Begin with 2×2 or 3×3 matrices to build geometric intuition about how U, Σ, and V^T work together before moving to larger matrices.

  4. Compare with Eigenvalues: For symmetric matrices, the SVD is closely related to the eigenvalue decomposition - singular values are the absolute values of eigenvalues.

  5. Look for Zero Singular Values: Zero singular values reveal the rank of the matrix and indicate dimensions that get completely collapsed by the transformation.