9.2.1 The Gram-Schmidt Process
In an inner product space, we can ask for bases that respect the geometry: bases whose vectors are mutually orthogonal and have unit length.
Definition 9.2.1 (Orthonormal basis). Let be a finite-dimensional inner product space. A basis of is orthonormal if Equivalently, for all , and for .
Note. In an orthonormal basis, the inner product takes a simple form: if and , then For real spaces, this is just . For complex spaces with Hermitian inner product, it is .
Theorem 9.2.1 (Existence of orthonormal bases). Every finite-dimensional inner product space has an orthonormal basis.
We give two proofs: one by induction, one constructive (Gram–Schmidt).
Proof 1 (Induction on dimension). Let . If , the empty set is an orthonormal basis. For :
Pick any nonzero and set , so . Let . By the orthogonal decomposition theorem, , and .
The restriction of to is still an inner product (check positive-definiteness). By induction, has an orthonormal basis .
Then is an orthonormal basis of : the are orthonormal in , and for since . ∎
Proof 2 (Gram–Schmidt). Start with any basis of . We inductively construct orthonormal vectors such that for each .
Step 1: Set .
Step : Given orthonormal , define This subtracts from its projections onto each , leaving a vector orthogonal to .
Since (linear independence), we have . Set
By construction, for , and . The span condition holds since and . ∎
Corollary 9.2.1 (Isomorphism with standard space). Every -dimensional real inner product space is isomorphic to as an inner product space. Similarly, every -dimensional Hermitian inner product space is isomorphic to .
Proof. Let be an orthonormal basis. The coordinate map sending to is a linear isomorphism. Moreover, which is the standard inner product of the coordinate vectors. ∎
Note. This says that all -dimensional inner product spaces (over the same field) are “the same.” The choice of orthonormal basis corresponds to the choice of isomorphism with standard space. Different orthonormal bases give different coordinate systems, all equally valid.
Example 9.2.1 (Gram–Schmidt in ). Apply Gram–Schmidt to the basis , , of .
Step 1: , so .
Step 2: . , so .
Step 3: Compute projections and proceed similarly to get .
9.2.2 Orthogonal Projections
The orthogonal decomposition gives a canonical way to project onto .
Definition 9.2.2 (Orthogonal projection). Let be an inner product space and a subspace. The orthogonal projection onto is the linear map defined by: for with and ,
Proposition 9.2.1 (Properties of orthogonal projections). Let be the orthogonal projection onto .
- and .
- (i.e., is idempotent).
- (i.e., is self-adjoint; see §9.3).
- .
Proof. (i) follows from the definition. For (ii): if , then , so . (iv) is immediate from . ∎
Proposition 9.2.2 (Formula in orthonormal basis). If is an orthonormal basis of , then
Proof. Let . We show : for any , Since spans , we have , so . Thus . ∎
Proposition 9.2.3 (Best approximation). For and a subspace, the projection is the unique vector in minimizing over .
Proof. For any , write where . Then: Since and , these are orthogonal: with equality iff . ∎