determin.ant
05-vector-spaces / 5.2

Module 5: Vector Spaces

Column, Null, and Row Space

Where you'll see this: in machine learning, the column space tells you which prediction targets the model can actually achieve. If your target isn't in the column space of your feature matrix, no combination of weights will reach it — this is why least squares finds the closest point instead.

Every matrix creates three fundamental spaces. Understanding them tells you everything about what the matrix can and can't do.

Column space (image)

The column space of A is the set of all vectors that A can produce — all outputs AxA\mathbf{x} for any input x. It's spanned by the columns of A.

Think of it as the "reach" of the matrix. If A is 2×2 with rank 2, it can reach anywhere in ℝ². If rank 1, it can only reach a line.

Null space (kernel)

The null space of A is all vectors x where Ax=0A\mathbf{x} = \mathbf{0}. These inputs get squashed to zero.

When A is full rank (rank = 2), the null space contains only the zero vector — the transformation is injective, no two inputs produce the same output. When rank drops, a whole line of inputs collapses to zero.

Rank-Nullity theorem: dim(column space) + dim(null space) = number of columns. For a 2×2 matrix: rank + nullity = 2. If rank = 1, then nullity = 1 (the null space is a line).

Row space

The row space is spanned by the rows of A. It lives in the input space and is the orthogonal complement of the null space. Together, the null space and row space partition every input vector perfectly.

Try it

Drag the column vectors. When they're linearly independent, the column space is all of ℝ² and the null space is just zero.

Drag them to be parallel — the matrix loses rank. The column space collapses to a line, and the null space becomes a line (shown in amber).

In NumPy:
import numpy as np
A = np.array([[2, 1], [1, 2]])

# Column space (via SVD)
U, s, Vh = np.linalg.svd(A)
rank = np.sum(s > 1e-9)

# Null space
_, _, Vh = np.linalg.svd(A)
null = Vh[rank:]  # rows of Vh after rank
Basis and DimensionRank
-6-6-5-5-4-4-3-3-2-2-1-1112233445566col₁col₂

Matrix A = [col₁ | col₂] =  [[2, 0], [1, 2]]

det(A) = 4

Column space = all of ℝ² · · Null space = just {0}

Drag the column vectors. Make them parallel to collapse the column space to a line.