Vector Spaces

One of the most important vector spaces is $\mathbf{R}^{n}$, containing real number vectors with $n$ elements. This is called an $n$-dimensional space. We have the first definition:

The space $\mathbf{R}^{n}$ contains all column vectors $v$ with $n$ elements.

We can perform addition between two vectors in the space $\mathbf{R}^{n}$ or multiply them by a scalar.

$ \begin{aligned} \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \\ \end{bmatrix} + \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \\ \end{bmatrix} = \begin{bmatrix} 2 \\ 3 \\ 4 \\ 5 \\ \end{bmatrix} ; \quad \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \\ \end{bmatrix} \times 2 = \begin{bmatrix} 2 \\ 4 \\ 6 \\ 8 \\ \end{bmatrix} \end{aligned} $

Above, we just performed addition of 2 vectors in the $\mathbf{R}^{4}$ space and multiplied a vector by a scalar. A note to remember is that the results of these operations must still be within the $\mathbf{R}^{4}$ space. Moreover, when performing calculations like the two expressions above, we also create linear combinations. There are 8 conditions to define a vector space. Let $v$ and $w$ be two vectors in the $\mathbf{R}^n$ space, let $c$ be a number, the addition $v+w$ and multiplication $cv$ must satisfy the following conditions:

  1. $v + w = w + v$
  2. $(v + w) + z = v + (w + z)$, where $z$ is a vector similar to $v$ and $w$
  3. $c(v+w) = cv + cw$, where $c$ is a number
  4. there must always exist a unique vector 0 such that $c + 0 = c$
  5. for every vector $v$, there always exists a unique vector $-v$ such that $v + (-v) = 0$
  6. $1 * v = v$
  7. $(c_{1} + c_{2})v = c_{1}v + c_{2}v$
  8. $(c_{1}c_{2})v = c_{1}(c_{2}v)$

Besides $\mathbf{R}^n$, we also have other vector spaces:

  • M: Space of real number matrices with a size of $2 \times 2$
  • F: Vector space of real functions $f(x)$
  • Z: Vector space containing only the zero vector

In M, vectors are matrices. Space F is an infinite-dimensional space, while in space Z the only addition performed is $0 + 0 = 0$. In each case, we can add: matrix to matrix, function to function, zero vector to zero vector. We can also multiply a term by a matrix, function, and zero vector. The results of these additions and multiplications naturally still lie within the M, F, and Z spaces. The 8 conditions mentioned above can also be proven to be satisfied. The Z space is the smallest space, as it only contains a single zero vector. Every vector space contains a zero vector: zero matrix, zero function, and zero vector $[0; 0; 0; 0]$ in the $\mathbf{R}^{4}$ space.

Subspaces

A subspace of $\mathbf{R}^{n}$ is a set of vector spaces within $\mathbf{R}^{n}$. For example, in the $\mathbf{R}^{3}$ space, a plane passing through the origin is a vector space. If we add any two vectors in the plane together or multiply a vector in the plane by a number, the resulting vector will still be in that plane. This leads to one of the most fundamental ideas in linear algebra, stated as follows:

A subspace is a set of vectors (including the zero vector) that satisfies two properties: if $v$ and $w$ are two vectors in the subspace and $c$ is a number, then

  • $v + w$ is in the subspace
  • $cv$ is in the subspace (and $dw$, where $d$ is also a number)

In other words, the vectors and their linear combinations are close to each other and all belong to the subspace. Because they are subsets of a larger space, the addition and multiplication operations still obey the 8 conditions defining a vector space. Additionally, subspaces all contain the zero vector. Considering the $\mathbf{R}^{3}$ space, any plane not passing through the origin $(0, 0, 0)$ is certainly not a subspace of $\mathbf{R}^{3}$. Furthermore, lines passing through the origin are also subspaces: when multiplying any number by a vector on a line, we obtain a new vector still on that line, and similarly for addition.

Lastly, we have some examples:

  • Subspaces of $\mathbf{R}^{3}$ include:

    • All vectors in $\mathbf{R}^{3}$
    • Any plane passing through the origin
    • Any line passing through the origin
    • The Z space (zero vector)

  • Subspaces of $\mathbf{R}^{2}$ include:

    • All vectors in $\mathbf{R}^{2}$
    • Any line passing through the origin
    • The $Z$ space (zero vector)

Column Space

This is the most important subspace directly associated with the matrix $A$. In linear algebra, to solve the equation $Ax = b$ if $A$ is not invertible, we can solve it for some $b$ and not for other $b$. If we find a good $b$, then $b$ can be written as $A$ times a vector $x$. These vectors $b$ are the column space of $A$.

To find the good vectors $b$ mentioned above, we have to find all linear combinations of the columns in $A$ (because $Ax$ is a combination of the columns in $A$, we have to find all possible vectors $x$). These linear combinations create the column space of $A$.

In summary, with $C(A)$ as the column space of $A$, then $C(A)$ will include not only the columns in $A$ but also all linear combinations $Ax$.

The column space contains all linear combinations of the columns in the matrix. These combinations are all the possible vectors $b$ = $Ax$.

Calling the column space the most important space is because, to solve $Ax = b$ means we are representing $b$ as a linear combination of $Ax$, or in other words, $b$ must lie within $C(A)$.

Suppose $A$ is a matrix of size $m \times n$, then each column of $A$ contains $m$ elements. Thus, the column space of $A$ is a subspace within the set $\mathbf{R}^m$ rather than $\mathbf{R}^n$. Additionally, combinations of $Ax$ must satisfy two laws for subspaces stated in the previous section.

Suppose we have a set of vectors $S$ in space $V$, to find the subspace $SS$ of $V$, similarly, we find all possible linear combinations of the vectors in set $S$.

$S = $ set of vectors belonging to $V$ (may not necessarily be a subspace)

$SS =$ linear combinations of vectors in $S$ (creates a subspace within $V$)

Then $SS$ is called the subspace of $V$ generated from $S$. This is also a basic way to generate a subspace.

If $S$ contains only a single vector $v$ other than the zero vector, the subspace $SS$ is a line passing through $v$. $SS$ always is the smallest subspace containing $S$.

An important note: considering $I$ as the identity vector in the $\mathbf{R}^{n}$ space ($I$ has $n$ rows $n$ columns), we have $C(I) =$ all vectors in $\mathbf{R}^{n}$. This is because all vectors are linear combinations of the columns in matrix $I$. You can easily prove this.