determin.ant — Linear Algebra from Scratch

Where you'll see this: SVD is behind image compression (JPEG-like low-rank approximations), recommender systems (Netflix-style matrix factorization), natural language processing (LSA/LSI), PCA, and pseudo-inverses. It's the most generally useful matrix decomposition in all of applied mathematics.

Every matrix does something to space. SVD reveals the exact structure of that something: every matrix is secretly a rotation, then a stretch, then another rotation.

The decomposition

Every matrix A can be written as:

A = U \Sigma V^T

Where:

Vᵀ is an orthogonal matrix (rotation/reflection in input space)
Σ is diagonal with non-negative entries (stretching)
U is an orthogonal matrix (rotation/reflection in output space)

The singular values

The diagonal entries σ₁ ≥ σ₂ ≥ ... ≥ 0 of Σ are the singular values. They tell you how much the matrix stretches space in each direction. A singular value of 0 means that direction collapses — that's a rank deficiency.

Think of A as doing three things in sequence to the unit circle:
1. Vᵀ rotates the circle (still a circle)
2. Σ stretches it into an ellipse
3. U rotates that ellipse
The final shape is an ellipse with semi-axes σ₁ and σ₂.

How it differs from eigendecomposition

Eigendecomposition only exists for square matrices and fails when eigenvalues are complex. SVD works for any matrix — rectangular, singular, complex. The singular values are always real and non-negative.

Try it

The four panels show the unit circle being transformed step by step. Use the sliders to adjust the two rotation angles and two singular values. Notice: the final shape depends only on σ₁ and σ₂ (the ellipse's size), while U and V control the orientation.

In NumPy:

import numpy as np
A = np.array([[2., 1.], [0., 1.5]])

# U: left singular vectors (columns)
# s: singular values (σ₁, σ₂)
# Vh: right singular vectors (rows = Vᵀ)
U, s, Vh = np.linalg.svd(A)

print(s)                         # [2.35, 1.06] — singular values
print(np.allclose(U @ np.diag(s) @ Vh, A))  # True

Rotate, Stretch, Rotate

The decomposition

The singular values

How it differs from eigendecomposition

Try it