Solutions to Review Question

by Qiang Gao, updated at Mar 20, 2017

Review Question 1.2.1

Prove that $\mathbf{X}' \mathbf{X}$ is positive definite if $\mathbf{X}$ is of full column rank.

Solution

By the definition of positive definite matrix, we need to show that $\mathbf{c}' \mathbf{X}' \mathbf{X} \mathbf{c} > 0$ for $\mathbf{c} \neq \mathbf{0}$. Define $\mathbf{z} \equiv \mathbf{X} \mathbf{c}$. Then $\mathbf{c}' \mathbf{X}' \mathbf{X} \mathbf{c} = \mathbf{z}' \mathbf{z} = \sum_{k=1}^{K} z_i^2$. If $\mathbf{X}$ is of full column rank, then the column vectors of $\mathbf{X}$ are linearly independent. This means $\mathbf{X} \mathbf{c} = \mathbf{0}$ if and only if $\mathbf{c} = \mathbf{0}$. Then $\mathbf{z} \neq \mathbf{0}$ for any $\mathbf{c} \neq \mathbf{0}$.

Copyright ©2017 by Qiang Gao