determin.ant
05-vector-spaces / 5.5

Module 5: Vector Spaces

Change of Basis

Where you'll see this: every time a 3D engine transforms a model from "object space" to "world space" to "camera space" to "screen space," it's doing change-of-basis operations. PCA also works by changing to the basis of eigenvectors of the covariance matrix — suddenly the data looks like independent, axis-aligned components.

A vector exists independently of any coordinate system. When you write [3, 2], you've implicitly chosen a basis. Change the basis, and the same vector gets different numbers — but the arrow in space doesn't move.

The formula

If your new basis vectors are the columns of matrix P, then the coordinates of vector v in the new basis are:

vnew=P1vstandard\mathbf{v}_{\text{new}} = P^{-1} \mathbf{v}_{\text{standard}}

And to convert back: vstandard=Pvnew\mathbf{v}_{\text{standard}} = P \mathbf{v}_{\text{new}}. P converts from new-basis coordinates to standard coordinates. P⁻¹ converts the other way.

Why it matters: similarity

Two matrices A and B represent the same linear transformation in different bases when:

B=P1APB = P^{-1} A P

This is called a similarity transformation. Diagonalization is exactly this — finding a basis (the eigenvector basis) where the matrix becomes diagonal.

Coordinates are just names for a vector in a particular language. Change of basis is switching languages — the object stays the same, only the description changes.

A key insight

The standard basis [1,0] and [0,1] is just one choice. Any two independent vectors form a valid basis. Picking the right basis — one aligned with the structure of your problem — often makes computations dramatically simpler. Eigenvectors, singular vectors, and Fourier modes are all "right" bases for specific problems.

Try it

Drag e₁ and e₂ to change the basis. The faint grid shows the new coordinate lines. Drag v to move the vector. Notice: [3, 2] in standard coordinates becomes something completely different in the new basis — same arrow, new numbers.

In NumPy:
import numpy as np

# New basis vectors as columns of P
P = np.array([[2., 1.], [0., 2.]])

v_standard = np.array([4., 2.])

# Convert to new basis
v_new = np.linalg.solve(P, v_standard)
print(v_new)  # coordinates in new basis

# Convert back
print(P @ v_new)  # should recover v_standard
Rank-Nullity TheoremEigenvectors — the Intuition
-6-6-5-5-4-4-3-3-2-2-1-1112233445566e₁e₂v

Standard basis

v = [3, 2]

New basis {e₁, e₂}

v = 1·e₁ + 1·e₂   →  [1, 1]new

The green grid lines show the new coordinate system. Same vector v, different numbers depending on the basis.