Skip to main content
Logo image

Understanding Linear Algebra (with Python)

David Austin, Adam Gilbert (Editor), Corrie Spaulding (Editor)

Section 2.2 Matrix multiplication and linear combinations

The previous section introduced vectors and linear combinations and demonstrated how they provide a way to think about linear systems geometrically. In particular, we saw that the vector \(\vec{b}\) is a linear combination of the vectors \(\vec{v_1},~\vec{v_2},\ldots,~\vec{v_n}\) precisely when the linear system corresponding to the augmented matrix
\begin{equation*} \left[\begin{array}{rrrr|r} \vec{v_1} \amp \vec{v_2} \amp \cdots \amp \vec{v_n} \amp \vec{b} \end{array}\right] \end{equation*}
is consistent.
🔗
Our goal in this section is to introduce matrix multiplication, another algebraic operation that deepens the connection between linear systems and linear combinations.
🔗

Subsection 2.2.1 Scalar multiplication and addition of matrices

We first thought of a matrix as a rectangular array of numbers. If we say that the shape of a matrix is \(m\times n\text{,}\) we mean that it has \(m\) rows and \(n\) columns. For instance, the shape of the matrix below is \(3\times4\text{:}\)
\begin{equation*} \left[ \begin{array}{rrrr} 0 \amp 4 \amp -3 \amp 1 \\ 3 \amp -1 \amp 2 \amp 0 \\ 2 \amp 0 \amp -1 \amp 1 \\ \end{array} \right]\text{.} \end{equation*}
🔗
We may also think of the columns of a matrix as a set of vectors. For instance, the matrix above may be represented as
\begin{equation*} \left[ \begin{array}{rrrr} \vec{v_1} \amp \vec{v_2} \amp \vec{v_3} \amp \vec{v_4} \end{array} \right] \end{equation*}
where
\begin{equation*} \vec{v_1}=\left[\begin{array}{r}0\\3\\2\\ \end{array}\right], \vec{v_2}=\left[\begin{array}{r}4\\-1\\0\\ \end{array}\right], \vec{v_3}=\left[\begin{array}{r}-3\\2\\-1\\ \end{array}\right], \vec{v_4}=\left[\begin{array}{r}1\\0\\1\\ \end{array}\right]\text{.} \end{equation*}
In this way, we see that the \(3\times 4\) matrix is equivalent to an ordered set of 4 vectors in \(\real^3\text{.}\)
🔗
This means that we may define scalar multiplication and matrix addition operations using the corresponding column-wise vector operations. For instance,
\begin{equation*} \begin{aligned} c\left[\begin{array}{rrrr} \vec{v_1} \amp \vec{v_2} \amp \cdots \amp \vec{v_n} \end{array} \right] {}={} \amp \left[\begin{array}{rrrr} c\vec{v_1} \amp c\vec{v_2} \amp \cdots \amp c\vec{v_n} \end{array} \right] \\ \left[\begin{array}{rrrr} \vec{v_1} \amp \vec{v_2} \amp \cdots \amp \vec{v_n} \end{array} \right] {}+{} \amp \left[\begin{array}{rrrr} \vec{w_1} \amp \vec{w_2} \amp \cdots \amp \vec{w_n} \end{array} \right] \\ {}={} \amp \left[\begin{array}{rrrr} \vec{v_1}+\vec{w_1} \amp \vec{v_2}+\vec{w_2} \amp \cdots \amp \vec{v_n}+\vec{w_n} \end{array} \right]. \\ \end{aligned} \end{equation*}
🔗

Preview Activity 2.2.1. Matrix operations.

  1. Compute the scalar multiple
    \begin{equation*} -3\left[ \begin{array}{rrr} 3 \amp 1 \amp 0 \\ -4 \amp 3 \amp -1 \\ \end{array} \right]\text{.} \end{equation*}
    🔗
    🔗
  2. Find the sum
    \begin{equation*} \left[ \begin{array}{rr} 0 \amp -3 \\ 1 \amp -2 \\ 3 \amp 4 \\ \end{array} \right] + \left[ \begin{array}{rrr} 4 \amp -1 \\ -2 \amp 2 \\ 1 \amp 1 \\ \end{array} \right]\text{.} \end{equation*}
    🔗
    🔗
  3. Suppose that \(A\) and \(B\) are two matrices. What do we need to know about their shapes before we can form the sum \(A+B\text{?}\)
    🔗
    🔗
  4. The matrix \(I_n\text{,}\) which we call the identity matrix, is the \(n\times n\) matrix whose entries are zero except for the diagonal entries, all of which are 1. For instance,
    \begin{equation*} I_3 = \left[ \begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \\ \end{array} \right]\text{.} \end{equation*}
    If we can form the sum \(A+I_n\text{,}\) what must be true about the matrix \(A\text{?}\)
    🔗
    🔗
  5. Find the matrix \(A - 2I_3\) where
    \begin{equation*} A = \left[ \begin{array}{rrr} 1 \amp 2 \amp -2 \\ 2 \amp -3 \amp 3 \\ -2 \amp 3 \amp 4 \\ \end{array} \right]\text{.} \end{equation*}
    🔗
    🔗
🔗
🔗
As this preview activity shows, the operations of scalar multiplication and addition of matrices are natural extensions of their vector counterparts. Some care, however, is required when adding matrices. Since we need the same number of vectors to add and since those vectors must be of the same dimension, two matrices must have the same shape if we wish to form their sum.
🔗
🔗

Subsection 2.2.2 Matrix-vector multiplication and linear combinations

A more important operation will be matrix multiplication as it allows us to compactly express linear systems. We now introduce the product of a matrix and a vector with an example.
🔗

Example 2.2.1. Matrix-vector multiplication.

Suppose we have the matrix \(A\) and vector \(\vec{x}\text{:}\)
\begin{equation*} A = \left[\begin{array}{rr} -2 \amp 3 \\ 0 \amp 2 \\ 3 \amp 1 \\ \end{array}\right],~~~ \vec{x} = \left[\begin{array}{r} 2 \\ 3 \\ \end{array}\right]\text{.} \end{equation*}
Their product will be defined to be the linear combination of the columns of \(A\) using the components of \(\vec{x}\) as weights. This means that
\begin{equation*} \begin{aligned} A\vec{x} = \left[\begin{array}{rr} -2 \amp 3 \\ 0 \amp 2 \\ 3 \amp 1 \\ \end{array}\right] \left[\begin{array}{r} 2 \\ 3 \\ \end{array}\right] {}={} \amp 2 \left[\begin{array}{r} -2 \\ 0 \\ 3 \\ \end{array}\right] + 3 \left[\begin{array}{r} 3 \\ 2 \\ 1 \\ \end{array}\right] \\ \\ {}={} \amp \left[\begin{array}{r} -4 \\ 0 \\ 6 \\ \end{array}\right] + \left[\begin{array}{r} 9 \\ 6 \\ 3 \\ \end{array}\right] \\ \\ {}={} \amp \left[\begin{array}{r} 5 \\ 6 \\ 9 \\ \end{array}\right]. \\ \end{aligned} \end{equation*}
🔗
Because \(A\) has two columns, we need two weights to form a linear combination of those columns, which means that \(\vec{x}\) must have two components. In other words, the number of columns of \(A\) must equal the dimension of the vector \(\vec{x}\text{.}\)
🔗
Similarly, the columns of \(A\) are 3-dimensional so any linear combination of them is 3-dimensional as well. Therefore, \(A\vec{x}\) will be 3-dimensional.
🔗
We then see that if \(A\) is a \(3\times2\) matrix, \(\vec{x}\) must be a 2-dimensional vector and \(A\vec{x}\) will be 3-dimensional.
🔗
🔗
More generally, we have the following definition.
🔗

Definition 2.2.2. Matrix-vector multiplication.

The product of a matrix \(A\) by a vector \(\vec{x}\) will be the linear combination of the columns of \(A\) using the components of \(\vec{x}\) as weights. More specifically, if
\begin{equation*} A=\left[\begin{array}{rrrr} \vec{v_1} \amp \vec{v_2} \amp \ldots \amp \vec{v_n} \end{array}\right],~~~ \vec{x} = \left[\begin{array}{r} c_1 \\ c_2 \\ \vdots \\ c_n \end{array}\right], \end{equation*}
then
\begin{equation*} A\vec{x} = c_1\vec{v_1} + c_2\vec{v_2} + \ldots + c_n\vec{v_n}\text{.} \end{equation*}
🔗
If \(A\) is an \(m\times n\) matrix, then \(\vec{x}\) must be an \(n\)-dimensional vector, and the product \(A\vec{x}\) will be an \(m\)-dimensional vector.
🔗
🔗
The next activity explores some properties of matrix multiplication.
🔗

Activity 2.2.2. Matrix-vector multiplication.

  1. Find the matrix product
    \begin{equation*} \left[ \begin{array}{rrrr} 1 \amp 2 \amp 0 \amp -1 \\ 2 \amp 4 \amp -3 \amp -2 \\ -1 \amp -2 \amp 6 \amp 1 \\ \end{array} \right] \left[ \begin{array}{r} 3 \\ 1 \\ -1 \\ 1 \\ \end{array} \right]\text{.} \end{equation*}
    🔗
    🔗
  2. Suppose that \(A\) is the matrix
    \begin{equation*} \left[ \begin{array}{rrr} 3 \amp -1 \amp 0 \\ 0 \amp -2 \amp 4 \\ 2 \amp 1 \amp 5 \\ 1 \amp 0 \amp 3 \\ \end{array} \right]\text{.} \end{equation*}
    If \(A\vec{x}\) is defined, what is the dimension of the vector \(\vec{x}\) and what is the dimension of \(A\vec{x}\text{?}\)
    🔗
    🔗
  3. A vector whose entries are all zero is denoted by \(\vec{0}\text{.}\) If \(A\) is a matrix, what is the product \(A\vec{0}\text{?}\)
    🔗
    🔗
  4. Suppose that \(I = \left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \\ \end{array}\right]\) is the identity matrix and \(\vec{x}=\threevec{x_1}{x_2}{x_3}\text{.}\) Find the product \(I\vec{x}\) and explain why \(I\) is called the identity matrix.
    🔗
    🔗
  5. Suppose we write the matrix \(A\) in terms of its columns as
    \begin{equation*} A = \left[ \begin{array}{rrrr} \vec{v_1} \amp \vec{v_2} \amp \cdots \amp \vec{v_n} \\ \end{array} \right]\text{.} \end{equation*}
    If the vector \(\vec{e_1} = \left[\begin{array}{c} 1 \\ 0 \\ \vdots \\ 0 \end{array}\right]\text{,}\) what is the product \(A\vec{e_1}\text{?}\)
    🔗
    🔗
  6. Suppose that
    \begin{equation*} A = \left[ \begin{array}{rrrr} 1 \amp 2 \\ -1 \amp 1 \\ \end{array} \right], \vec{b} = \left[ \begin{array}{r} 6 \\ 0 \end{array} \right]\text{.} \end{equation*}
    Is there a vector \(\vec{x}\) such that \(A\vec{x} = \vec{b}\text{?}\)
    🔗
    🔗
🔗
🔗
Multiplication of a matrix \(A\) and a vector is defined as a linear combination of the columns of \(A\text{.}\) However, there is a shortcut for computing such a product. Let’s look at our previous example and focus on the first row of the product.
\begin{equation*} \left[\begin{array}{rr} -2 \amp 3 \\ 0 \amp 2 \\ 3 \amp 1 \\ \end{array}\right] \left[\begin{array}{r} 2 \\ 3 \\ \end{array}\right] = 2 \left[\begin{array}{r} -2 \\ * \\ * \\ \end{array}\right] + 3 \left[\begin{array}{r} 3 \\ * \\ * \\ \end{array}\right] = \left[\begin{array}{c} 2(-2)+3(3) \\ * \\ * \\ \end{array}\right] = \left[\begin{array}{r} 5 \\ * \\ * \\ \end{array}\right]\text{.} \end{equation*}
🔗
To find the first component of the product, we consider the first row of the matrix. We then multiply the first entry in that row by the first component of the vector, the second entry by the second component of the vector, and so on, and add the results. In this way, we see that the third component of the product would be obtained from the third row of the matrix by computing \(2(3) + 3(1) = 9\text{.}\)
🔗
You are encouraged to evaluate the product Item a of the previous activity using this shortcut and compare the result to what you found while completing that activity.
🔗

Activity 2.2.3.

In Python we can use * to find the product between two compatible matrices or a matrix and compatible vector. For example, we can compute the product \(\left[\begin{array}{ll} 1 \amp 2\\ 2 \amp 1\end{array}\right]\left[\begin{array}{l} 3\\ -1\end{array}\right]\) as follows.
  1. Use Python to evaluate the product
    \begin{equation*} \left[ \begin{array}{rrrr} 1 \amp 2 \amp 0 \amp -1 \\ 2 \amp 4 \amp -3 \amp -2 \\ -1 \amp -2 \amp 6 \amp 1 \\ \end{array} \right] \left[ \begin{array}{r} 3 \\ 1 \\ -1 \\ 1 \\ \end{array} \right] \end{equation*}
    from Item a of the previous activity.
    🔗
    🔗
  2. In Python, define the matrix and vectors
    \begin{equation*} A = \left[ \begin{array}{rrr} -2 \amp 0 \\ 3 \amp 1 \\ 4 \amp 2 \\ \end{array} \right], \vec{0} = \left[ \begin{array}{r} 0 \\ 0 \end{array} \right], \vec{v} = \left[ \begin{array}{r} -2 \\ 3 \end{array} \right], \vec{w} = \left[ \begin{array}{r} 1 \\ 2 \end{array} \right]\text{.} \end{equation*}
    🔗
    🔗
  3. What do you find when you evaluate \(A\vec{0}\text{?}\)
    🔗
    🔗
  4. What do you find when you evaluate \(A(3\vec{v})\) and \(3(A\vec{v})\) and compare your results?
    🔗
    🔗
  5. What do you find when you evaluate \(A(\vec{v}+\vec{w})\) and \(A\vec{v} + A\vec{w}\) and compare your results?
    🔗
    🔗
🔗
🔗
This activity demonstrates several general properties satisfied by matrix multiplication that we record here.
🔗
🔗

Subsection 2.2.3 Matrix-vector multiplication and linear systems

So far, we have begun with a matrix \(A\) and a vector \(\vec{x}\) and formed their product \(A\vec{x} = \vec{b}\text{.}\) We would now like to turn this around: beginning with a matrix \(A\) and a vector \(\vec{b}\text{,}\) we will ask if we can find a vector \(\vec{x}\) such that \(A\vec{x} = \vec{b}\text{.}\) This will naturally lead back to linear systems.
🔗
To see the connection between the matrix equation \(A\vec{x} = \vec{b}\) and linear systems, let’s write the matrix \(A\) in terms of its columns \(\vec{v_i}\) and \(\vec{x}\) in terms of its components.
\begin{equation*} A = \left[ \begin{array}{rrrr} \vec{v_1} \amp \vec{v_2} \amp \ldots \vec{v_n} \end{array} \right], \vec{x} = \left[ \begin{array}{c} c_1 \\ c_2 \\ \vdots \\ c_n \\ \end{array} \right]\text{.} \end{equation*}
🔗
We know that the matrix product \(A\vec{x}\) forms a linear combination of the columns of \(A\text{.}\) Therefore, the equation \(A\vec{x} = \vec{b}\) is merely a compact way of writing the equation for the weights \(c_i\text{:}\)
\begin{equation*} c_1\vec{v_1} + c_2\vec{v_2} + \ldots + c_n\vec{v_n} = \vec{b}\text{.} \end{equation*}
We have seen this equation before: Remember that Proposition 2.1.12 says that the solutions of this equation are the same as the solutions to the linear system whose augmented matrix is
\begin{equation*} \left[\begin{array}{rrrr|r} \vec{v_1} \amp \vec{v_2} \amp \ldots \amp \vec{v_n} \amp \vec{b} \end{array}\right]\text{.} \end{equation*}
🔗
This gives us three different ways of looking at the same solution space.
🔗
When the matrix \(A = \left[\begin{array}{rrrr} \vec{v_1}\amp\vec{v_2}\amp\cdots\amp\vec{v_n}\end{array}\right]\text{,}\) we will frequently write
\begin{equation*} \left[\begin{array}{rrrr|r} \vec{v_1}\amp\vec{v_2}\amp\cdots\amp\vec{v_n}\amp\vec{b}\end{array}\right] = \left[ \begin{array}{r|r} A \amp \vec{b} \end{array}\right] \end{equation*}
and say that the matrix \(A\) is augmented by the vector \(\vec{b}\text{.}\)
🔗
The equation \(A\vec{x} = \vec{b}\) gives a notationally compact way to write a linear system. Moreover, this notation will allow us to focus on important features of the system that determine its solution space.
🔗

Example 2.2.5.

We will describe the solution space of the equation
\begin{equation*} \left[\begin{array}{rrr} 2 \amp 0 \amp 2 \\ 4 \amp -1 \amp 6 \\ 1 \amp 3 \amp -5 \\ \end{array}\right] \vec{x} = \left[\begin{array}{r} 0 \\ -5 \\ 15 \end{array}\right]. \end{equation*}
🔗
By Proposition 2.2.4, this equation may be equivalently expressed as
\begin{equation*} x_1\left[\begin{array}{r}2\\4\\1\end{array}\right] + x_2\left[\begin{array}{r}0\\-1\\3\end{array}\right]+ x_3\left[\begin{array}{r}2\\6\\-5\end{array}\right]= \left[\begin{array}{r}0\\-5\\15\end{array}\right]\text{,} \end{equation*}
which is the linear system corresponding to the augmented matrix
\begin{equation*} \left[\begin{array}{rrr|r} 2 \amp 0 \amp 2 \amp 0 \\ 4 \amp -1 \amp 6 \amp -5 \\ 1 \amp 3 \amp -5 \amp 15 \\ \end{array} \right]\text{.} \end{equation*}
The reduced row echelon form of the augmented matrix is
\begin{equation*} \left[\begin{array}{rrr|r} 2 \amp 0 \amp 2 \amp 0 \\ 4 \amp -1 \amp 6 \amp -5 \\ 1 \amp 3 \amp -5 \amp 15 \\ \end{array} \right] \sim \left[\begin{array}{rrr|r} 1 \amp 0 \amp 1 \amp 0 \\ 0 \amp 1 \amp -2 \amp 5 \\ 0 \amp 0 \amp 0 \amp 0 \\ \end{array} \right], \end{equation*}
which corresponds to the linear system
\begin{equation*} \begin{alignedat}{4} x_1 \amp \amp \amp {}+{} \amp x_3 \amp {}={} \amp 0 \\ \amp \amp x_2 \amp {}-{} \amp 2x_3 \amp {}={} \amp 5. \\ \end{alignedat} \end{equation*}
The variable \(x_3\) is free so we may write the solution space parametrically as
\begin{equation*} \begin{aligned} x_1 \amp {}={} -x_3 \\ x_2 \amp {}={} 5+2x_3. \\ \end{aligned} \end{equation*}
🔗
Since we originally asked to describe the solutions to the equation \(A\vec{x} = \vec{b}\text{,}\) we will express the solution in terms of the vector \(\vec{x}\text{:}\)
\begin{equation*} \vec{x} =\left[ \begin{array}{r} x_1 \\ x_2 \\ x_3 \end{array} \right] = \left[ \begin{array}{r} -x_3 \\ 5 + 2x_3 \\ x_3 \end{array} \right] =\left[\begin{array}{r}0\\5\\0\end{array}\right] +x_3\left[\begin{array}{r}-1\\2\\1\end{array}\right] \end{equation*}
As before, we call this a parametric description of the solution space.
🔗
This shows that the solutions \(\vec{x}\) may be written in the form \(\vec{v} + x_3\vec{w}\text{,}\) for appropriate vectors \(\vec{v}\) and \(\vec{w}\text{.}\) Geometrically, the solution space is a line in \(\real^3\) through \(\vec{v}\) moving parallel to \(\vec{w}\text{.}\)
🔗
🔗

Activity 2.2.4. The equation \(A\vec{x} = \vec{b}\).

  1. Consider the linear system
    \begin{equation*} \begin{alignedat}{4} 2x \amp {}+{} \amp y \amp {}-{} \amp 3z \amp {}={} \amp 4 \\ -x \amp {}+{} \amp 2y \amp {}+{} \amp z \amp {}={} \amp 3 \\ 3x \amp {}-{} \amp y \amp \amp \amp {}={} \amp -4. \\ \end{alignedat} \end{equation*}
    Identify the matrix \(A\) and vector \(\vec{b}\) to express this system in the form \(A\vec{x} = \vec{b}\text{.}\)
    🔗
    🔗
  2. If \(A\) and \(\vec{b}\) are as below, write the linear system corresponding to the equation \(A\vec{x}=\vec{b}\) and describe its solution space, using a parametric description if appropriate:
    \begin{equation*} A = \left[\begin{array}{rrr} 3 \amp -1 \amp 0 \\ -2 \amp 0 \amp 6 \end{array} \right],~~~ \vec{b} = \left[\begin{array}{r} -6 \\ 2 \end{array} \right]. \end{equation*}
    🔗
    🔗
  3. Describe the solution space of the equation
    \begin{equation*} \left[ \begin{array}{rrrr} 1 \amp 2 \amp 0 \amp -1 \\ 2 \amp 4 \amp -3 \amp -2 \\ -1 \amp -2 \amp 6 \amp 1 \\ \end{array} \right] \vec{x} = \left[\begin{array}{r} -1 \\ 1 \\ 5 \end{array} \right]\text{.} \end{equation*}
    🔗
    🔗
  4. Suppose \(A\) is an \(m\times n\) matrix. What can you guarantee about the solution space of the equation \(A\vec{x} = \vec{0}\text{?}\)
    🔗
    🔗
🔗
🔗
🔗

Subsection 2.2.4 Matrix-matrix products

In this section, we have developed some algebraic operations on matrices with the aim of simplifying our description of linear systems. We now introduce a final operation, the product of two matrices, that will become important when we study linear transformations in Section 2.5.
🔗

Definition 2.2.6. Matrix-matrix multiplication.

Given matrices \(A\) and \(B\text{,}\) we form their product \(AB\) by first writing \(B\) in terms of its columns
\begin{equation*} B = \left[\begin{array}{rrrr} \vec{v_1} \amp \vec{v_2} \amp \cdots \amp \vec{v_p} \end{array}\right] \end{equation*}
and then defining
\begin{equation*} AB = \left[\begin{array}{rrrr} A\vec{v_1} \amp A\vec{v_2} \amp \cdots \amp A\vec{v_p} \end{array}\right]. \end{equation*}
🔗
🔗

Example 2.2.7.

Given the matrices
\begin{equation*} A = \left[\begin{array}{rr} 4 \amp 2 \\ 0 \amp 1 \\ -3 \amp 4 \\ 2 \amp 0 \\ \end{array}\right],~~~ B = \left[\begin{array}{rrr} -2 \amp 3 \amp 0 \\ 1 \amp 2 \amp -2 \\ \end{array}\right]\text{,} \end{equation*}
we have
\begin{equation*} AB = \left[\begin{array}{rrr} A \twovec{-2}{1} \amp A \twovec{3}{2} \amp A \twovec{0}{-2} \end{array}\right] = \left[\begin{array}{rrr} -6 \amp 16 \amp -4 \\ 1 \amp 2 \amp -2 \\ 10 \amp -1 \amp -8 \\ -4 \amp 6 \amp 0 \end{array}\right]\text{.} \end{equation*}
🔗
🔗

Observation 2.2.8.

It is important to note that we can only multiply matrices if the shapes of the matrices are compatible. More specifically, when constructing the product \(AB\text{,}\) the matrix \(A\) multiplies the columns of \(B\text{.}\) Therefore, the number of columns of \(A\) must equal the number of rows of \(B\text{.}\) When this condition is met, the number of rows of \(AB\) is the number of rows of \(A\text{,}\) and the number of columns of \(AB\) is the number of columns of \(B\text{.}\)
🔗
🔗

Activity 2.2.5.

Consider the matrices
\begin{equation*} A = \left[\begin{array}{rrr} 1 \amp 3 \amp 2 \\ -3 \amp 4 \amp -1 \\ \end{array}\right],~~~ B = \left[\begin{array}{rr} 3 \amp 0 \\ 1 \amp 2 \\ -2 \amp -1 \\ \end{array}\right]\text{.} \end{equation*}
  1. Before computing, first explain why the shapes of \(A\) and \(B\) enable us to form the product \(AB\text{.}\) Then describe the shape of \(AB\text{.}\)
    🔗
    🔗
  2. Compute the product \(AB\text{.}\)
    🔗
    🔗
  3. Like with matrix-vector multiplication, we can use * to find the product of two compatible matrices (as long as they are defined with {sympy}). Define the matrices \(A\) and \(B\) in the Python cell below and check your work by computing \(AB\text{.}\)
    🔗
    🔗
  4. Are we able to form the matrix product \(BA\text{?}\) If so, use the Python cell above to find \(BA\text{.}\) Is it generally true that \(AB = BA\text{?}\)
    🔗
    🔗
  5. Suppose we form the three matrices.
    \begin{equation*} A = \left[\begin{array}{rr} 1 \amp 2 \\ 3 \amp -2 \\ \end{array}\right], B = \left[\begin{array}{rr} 0 \amp 4 \\ 2 \amp -1 \\ \end{array}\right], C = \left[\begin{array}{rr} -1 \amp 3 \\ 4 \amp 3 \\ \end{array}\right]\text{.} \end{equation*}
    Compare what happens when you compute \(A(B+C)\) and \(AB + AC\text{.}\) State your finding as a general principle.
    🔗
    🔗
  6. Compare the results of evaluating \(A(BC)\) and \((AB)C\) and state your finding as a general principle.
    🔗
    🔗
  7. When we are dealing with real numbers, we know if \(a\neq 0\) and \(ab = ac\text{,}\) then \(b=c\text{.}\) Define matrices
    \begin{equation*} A = \left[\begin{array}{rr} 1 \amp 2 \\ -2 \amp -4 \\ \end{array}\right], B = \left[\begin{array}{rr} 3 \amp 0 \\ 1 \amp 3 \\ \end{array}\right], C = \left[\begin{array}{rr} 1 \amp 2 \\ 2 \amp 2 \\ \end{array}\right] \end{equation*}
    and compute \(AB\) and \(AC\text{.}\)
    If \(AB = AC\text{,}\) is it necessarily true that \(B = C\text{?}\)
    🔗
    🔗
  8. Again, with real numbers, we know that if \(ab = 0\text{,}\) then either \(a = 0\) or \(b=0\text{.}\) Define
    \begin{equation*} A = \left[\begin{array}{rr} 1 \amp 2 \\ -2 \amp -4 \\ \end{array}\right], B = \left[\begin{array}{rr} 2 \amp -4 \\ -1 \amp 2 \\ \end{array}\right] \end{equation*}
    and compute \(AB\text{.}\)
    If \(AB = 0\text{,}\) is it necessarily true that either \(A=0\) or \(B=0\text{?}\)
    🔗
    🔗
🔗
🔗
This activity demonstrated some general properties about products of matrices, which mirror some properties about operations with real numbers.
🔗

Properties of Matrix-matrix Multiplication.

If \(A\text{,}\) \(B\text{,}\) and \(C\) are matrices such that the following operations are defined, it follows that
Associativity:
🔗
\(A(BC) = (AB)C\text{.}\)
🔗
Distributivity:
🔗
\(A(B+C) = AB+AC\text{.}\)
🔗
\((A+B)C = AC+BC\text{.}\)
🔗
🔗
🔗
At the same time, there are a few properties that hold for real numbers that do not hold for matrices.
🔗

Caution.

The following properties hold for real numbers but not for matrices.
Commutativity:
🔗
It is not generally true that \(AB = BA\text{.}\)
🔗
Cancellation:
🔗
It is not generally true that \(AB = AC\) implies that \(B = C\text{.}\)
🔗
Zero divisors:
🔗
It is not generally true that \(AB = 0\) implies that either \(A=0\) or \(B=0\text{.}\)
🔗
🔗
🔗
🔗

Subsection 2.2.5 Summary

In this section, we have found an especially simple way to express linear systems using matrix multiplication.
  • If \(A\) is an \(m\times n\) matrix and \(\vec{x}\) an \(n\)-dimensional vector, then \(A\vec{x}\) is the linear combination of the columns of \(A\) using the components of \(\vec{x}\) as weights. The vector \(A\vec{x}\) is \(m\)-dimensional.
    🔗
    🔗
  • The solution space to the equation \(A\vec{x} = \vec{b}\) is the same as the solution space to the linear system corresponding to the augmented matrix \(\left[ \begin{array}{r|r} A \amp \vec{b} \end{array}\right]\text{.}\)
    🔗
    🔗
  • If \(A\) is an \(m\times n\) matrix and \(B\) is an \(n\times p\) matrix, we can form the product \(AB\text{,}\) which is an \(m\times p\) matrix whose columns are the products of \(A\) and the columns of \(B\text{.}\)
    🔗
    🔗
🔗
🔗

Exercises 2.2.6 Exercises

1.

Consider the system of linear equations
\begin{equation*} \begin{alignedat}{4} x \amp {}+{} \amp 2y \amp {}-{} \amp z \amp {}={} \amp 1 \\ 3x \amp {}+{} \amp 2y \amp {}+{} \amp 2z \amp {}={} \amp 7 \\ -x \amp \amp \amp {}+{} \amp 4z \amp {}={} \amp -3 \\ \end{alignedat}\text{.} \end{equation*}
  1. Find the matrix \(A\) and vector \(\vec{b}\) that expresses this linear system in the form \(A\vec{x}=\vec{b}\text{.}\)
    🔗
    🔗
  2. Give a description of the solution space to the equation \(A\vec{x} = \vec{b}\text{.}\)
    🔗
    🔗
🔗
🔗

2.

Suppose that \(A\) is a \(135\times2201\) matrix, and that \(\vec{x}\) is a vector. If \(A\vec{x}\) is defined, what is the dimension of \(\vec{x}\text{?}\) What is the dimension of \(A\vec{x}\text{?}\)
🔗
🔗

3.

Suppose that \(A \) is a \(3\times2\) matrix whose columns are \(\vec{v_1}\) and \(\vec{v_2}\text{;}\) that is,
\begin{equation*} A = \left[\begin{array}{rr} \vec{v_1} \amp \vec{v_2} \end{array} \right]\text{.} \end{equation*}
  1. What is the dimension of the vectors \(\vec{v_1}\) and \(\vec{v_2}\text{?}\)
    🔗
    🔗
  2. What is the product \(A\twovec{1}{0}\) in terms of \(\vec{v_1}\) and \(\vec{v_2}\text{?}\) What is the product \(A\twovec{0}{1}\text{?}\) What is the product \(A\twovec{2}{3}\text{?}\)
    🔗
    🔗
  3. If we know that
    \begin{equation*} A\twovec{1}{0} = \threevec{3}{-2}{1},~~~ A\twovec{0}{1} = \threevec{0}{3}{2}, \end{equation*}
    what is the matrix \(A\text{?}\)
    🔗
    🔗
🔗
🔗

4.

Suppose that the matrix \(A = \left[\begin{array}{rr} \vec{v_1} \amp \vec{v_2} \end{array}\right]\) where \(\vec{v_1}\) and \(\vec{v_2}\) are shown in Figure 2.2.9.
🔗
Figure 2.2.9. Two vectors \(\vec{v_1}\) and \(\vec{v_2}\) that form the columns of the matrix \(A\text{.}\)
🔗
  1. What is the shape of the matrix \(A\text{?}\)
    🔗
    🔗
  2. On Figure 2.2.9, indicate the vectors
    \begin{equation*} A\twovec{1}{0}, ~~~A\twovec{2}{3}, ~~~A\twovec{0}{-3}\text{.} \end{equation*}
    🔗
    🔗
  3. Find all vectors \(\vec{x}\) such that \(A\vec{x}=\vec{b}\text{.}\)
    🔗
    🔗
  4. Find all vectors \(\vec{x}\) such that \(A\vec{x} = \vec{0}\text{.}\)
    🔗
    🔗
🔗
🔗

5.

Suppose that
\begin{equation*} A=\left[\begin{array}{rrr} 1 \amp 0 \amp 2 \\ 2 \amp 2 \amp 2 \\ -1 \amp -3 \amp 1 \end{array}\right]\text{.} \end{equation*}
  1. Describe the solution space to the equation \(A\vec{x} = \vec{0}\text{.}\)
    🔗
    🔗
  2. Find a \(3\times2\) matrix \(B\) with no zero entries such that \(AB = 0\text{.}\)
    🔗
    🔗
🔗
🔗

6.

Consider the matrix
\begin{equation*} A=\left[\begin{array}{rrrr} 1 \amp 2 \amp -4 \amp -4 \\ 2 \amp 3 \amp 0 \amp 1 \\ 1 \amp 0 \amp 4 \amp 6 \\ \end{array}\right]\text{.} \end{equation*}
  1. Find the product \(A\vec{x}\) where
    \begin{equation*} \vec{x} = \fourvec{1}{-2}{0}{2}\text{.} \end{equation*}
    🔗
    🔗
  2. Give a description of the vectors \(\vec{x}\) such that
    \begin{equation*} A\vec{x} = \threevec{-1}{15}{17}\text{.} \end{equation*}
    🔗
    🔗
  3. Find the reduced row echelon form of \(A\) and identify the pivot positions.
    🔗
    🔗
  4. Can you find a vector \(\vec{b}\) such that \(A\vec{x}=\vec{b}\) is inconsistent?
    🔗
    🔗
  5. For a general 3-dimensional vector \(\vec{b}\text{,}\) what can you say about the solution space of the equation \(A\vec{x} = \vec{b}\text{?}\)
    🔗
    🔗
🔗
🔗

7.

The operations that we perform in Gaussian elimination can be accomplished using matrix multiplication. This observation is the basis of an important technique that we will investigate in a subsequent chapter.
🔗
Let’s consider the matrix
\begin{equation*} A = \left[\begin{array}{rrr} 1 \amp 2 \amp -1 \\ 2 \amp 0 \amp 2 \\ -3 \amp 2 \amp 3 \\ \end{array}\right]\text{.} \end{equation*}
  1. Suppose that
    \begin{equation*} S = \left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 7 \amp 0 \\ 0 \amp 0 \amp 1 \\ \end{array}\right]\text{.} \end{equation*}
    Verify that \(SA\) is the matrix that results when the second row of \(A\) is scaled by a factor of 7. What matrix \(S\) would scale the third row by -3?
    🔗
    🔗
  2. Suppose that
    \begin{equation*} P = \left[\begin{array}{rrr} 0 \amp 1 \amp 0 \\ 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 1 \\ \end{array}\right]\text{.} \end{equation*}
    Verify that \(PA\) is the matrix that results from interchanging the first and second rows. What matrix \(P\) would interchange the first and third rows?
    🔗
    🔗
  3. Suppose that
    \begin{equation*} L_1 = \left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ -2 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \\ \end{array}\right]\text{.} \end{equation*}
    Verify that \(L_1A\) is the matrix that results from multiplying the first row of \(A\) by \(-2\) and adding it to the second row. What matrix \(L_2\) would multiply the first row by 3 and add it to the third row?
    🔗
    🔗
  4. When we performed Gaussian elimination, our first goal was to perform row operations that brought the matrix into a triangular form. For our matrix \(A\text{,}\) find the row operations needed to find a row equivalent matrix \(U\) in triangular form. By expressing these row operations in terms of matrix multiplication, find a matrix \(L\) such that \(LA = U\text{.}\)
    🔗
    🔗
🔗
🔗

8.

In this exercise, you will construct the inverse of a matrix, a subject that we will investigate more fully in the next chapter. Suppose that \(A\) is the \(2\times2\) matrix:
\begin{equation*} A = \left[\begin{array}{rr} 3 \amp -2 \\ -2 \amp 1 \\ \end{array}\right]\text{.} \end{equation*}
  1. Find the vectors \(\vec{b}_1\) and \(\vec{b}_2\) such that the matrix \(B=\left[\begin{array}{rr} \vec{b}_1 \amp \vec{b}_2 \end{array}\right]\) satisfies
    \begin{equation*} AB = I = \left[\begin{array}{rr} 1 \amp 0 \\ 0 \amp 1 \\ \end{array}\right]\text{.} \end{equation*}
    🔗
    🔗
  2. In general, it is not true that \(AB = BA\text{.}\) Check that it is true, however, for the specific \(A\) and \(B\) that appear in this problem.
    🔗
    🔗
  3. Suppose that \(\vec{x} = \twovec{x_1}{x_2}\text{.}\) What do you find when you evaluate \(I\vec{x}\text{?}\)
    🔗
    🔗
  4. Suppose that we want to solve the equation \(A\vec{x} = \vec{b}\text{.}\) We know how to do this using Gaussian elimination; let’s use our matrix \(B\) to find a different way:
    \begin{equation*} \begin{aligned} A\vec{x} \amp {}={} \vec{b} \\ B(A\vec{x}) \amp {}={} B\vec{b} \\ (BA)\vec{x} \amp {}={} B\vec{b} \\ I\vec{x} \amp {}={} B\vec{b} \\ \vec{x} \amp {}={} B\vec{b} \\ \end{aligned}\text{.} \end{equation*}
    In other words, the solution to the equation \(A\vec{x}=\vec{b}\) is \(\vec{x} = B\vec{b}\text{.}\)
    🔗
    Consider the equation \(A\vec{x} = \twovec{5}{-2}\text{.}\) Find the solution in two different ways, first using Gaussian elimination and then as \(\vec{x} = B\vec{b}\text{,}\) and verify that you have found the same result.
    🔗
    🔗
🔗
🔗

9.

Determine whether the following statements are true or false and provide a justification for your response.
  1. If \(A\vec{x}\) is defined, then the number of components of \(\vec{x}\) equals the number of rows of \(A\text{.}\)
    🔗
    🔗
  2. The solution space to the equation \(A\vec{x} = \vec{b}\) is equivalent to the solution space to the linear system whose augmented matrix is \(\left[\begin{array}{r|r} A \amp \vec{b} \end{array}\right]\text{.}\)
    🔗
    🔗
  3. If a linear system of equations has 8 equations and 5 unknowns, then the shape of the matrix \(A\) in the corresponding equation \(A\vec{x} = \vec{b}\) is \(5\times8\text{.}\)
    🔗
    🔗
  4. If \(A\) has a pivot position in every row, then every equation \(A\vec{x} = \vec{b}\) is consistent.
    🔗
    🔗
  5. If \(A\) is a \(9\times5\) matrix, then \(A\vec{x}=\vec{b}\) is inconsistent for some vector \(\vec{b}\text{.}\)
    🔗
    🔗
🔗
🔗

10.

Suppose that \(A\) is a \(4\times4\) matrix and that the equation \(A\vec{x} = \vec{b}\) has a unique solution for some vector \(\vec{b}\text{.}\)
  1. What does this say about the pivot positions of the matrix \(A\text{?}\) Write the reduced row echelon form of \(A\text{.}\)
    🔗
    🔗
  2. Can you find another vector \(\vec{c}\) such that \(A\vec{x} = \vec{c}\) is inconsistent?
    🔗
    🔗
  3. What can you say about the solution space to the equation \(A\vec{x} = \vec{0}\text{?}\)
    🔗
    🔗
  4. Suppose \(A=\left[\begin{array}{rrrr} \vec{v_1} \amp \vec{v_2} \amp \vec{v_3} \amp \vec{v_4} \end{array}\right]\text{.}\) Explain why every four-dimensional vector can be written as a linear combination of the vectors \(\vec{v_1}\text{,}\) \(\vec{v_2}\text{,}\) \(\vec{v_3}\text{,}\) and \(\vec{v_4}\) in exactly one way.
    🔗
    🔗
🔗
🔗

11.

Define the matrix
\begin{equation*} A = \left[\begin{array}{rrr} 1 \amp 2 \amp 4 \\ -2 \amp 1 \amp -3 \\ 3 \amp 1 \amp 7 \\ \end{array}\right]\text{.} \end{equation*}
  1. Describe the solution space to the homogeneous equation \(A\vec{x} = \vec{0}\) using a parametric description, if appropriate. What does this solution space represent geometrically?
    🔗
    🔗
  2. Describe the solution space to the equation \(A\vec{x}=\vec{b}\) where \(\vec{b} = \threevec{-3}{-4}{1}\text{.}\) What does this solution space represent geometrically and how does it compare to the previous solution space?
    🔗
    🔗
  3. We will now explain the relationship between the previous two solution spaces. Suppose that \(\vec{x_h}\) is a solution to the homogeneous equation; that is \(A\vec{x_h}=\vec{0}\text{.}\) Suppose also that \(\vec{x_p}\) is a solution to the equation \(A\vec{x} = \vec{b}\text{;}\) that is, \(A\vec{x_p}=\vec{b}\text{.}\)
    🔗
    Use the Linearity Principle expressed in Proposition 2.2.3 to explain why \(\vec{x_h}+\vec{x_p}\) is a solution to the equation \(A\vec{x} = \vec{b}\text{.}\) You may do this by evaluating \(A(\vec{x_h}+\vec{x_p})\text{.}\)
    🔗
    That is, if we find one solution \(\vec{x_p}\) to an equation \(A\vec{x} = \vec{b}\text{,}\) we may add any solution to the homogeneous equation to \(\vec{x_p}\) and still have a solution to the equation \(A\vec{x} = \vec{b}\text{.}\) In other words, the solution space to the equation \(A\vec{x} = \vec{b}\) is given by translating the solution space to the homogeneous equation by the vector \(\vec{x_p}\text{.}\)
    🔗
    🔗
🔗
🔗

12.

Suppose that a city is starting a bicycle sharing program with bicycles at locations \(B\) and \(C\text{.}\) Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(B\) are returned to \(B\) with the other 20% returned to \(C\text{.}\) Similarly, 50% of bicycles rented at location \(C\) are returned to \(B\) and 50% to \(C\text{.}\)
🔗
To keep track of the bicycles, we form a vector
\begin{equation*} \vec{x_k} = \twovec{B_k}{C_k} \end{equation*}
where \(B_k\) is the number of bicycles at location \(B\) at the beginning of day \(k\) and \(C_k\) is the number of bicycles at \(C\text{.}\) The information above tells us how to determine the distribution of bicycles the following day:
\begin{equation*} \begin{alignedat}{3} B_{k+1} \amp {}={} 0.8B_k \amp {}+{} \amp 0.5 C_k \\ C_{k+1} \amp {}={} 0.2B_k \amp {}+{} \amp 0.5 C_k. \\ \end{alignedat} \end{equation*}
Expressed in matrix-vector form, these expressions give
\begin{equation*} \vec{x_{k+1}} = A\vec{x_k} \end{equation*}
where
\begin{equation*} A = \left[\begin{array}{rr} 0.8 \amp 0.5 \\ 0.2 \amp 0.5 \\ \end{array}\right]\text{.} \end{equation*}
  1. Let’s check that this makes sense.
    1. Suppose that there are 1000 bicycles at location \(B\) and none at \(C\) on day 1. This means we have \(\vec{x_1} = \twovec{1000}{0}\text{.}\) Find the number of bicycles at both locations on day 2 by evaluating \(\vec{x_2} = A\vec{x_1}\text{.}\)
      🔗
      🔗
    2. Suppose that there are 1000 bicycles at location \(C\) and none at \(B\) on day 1. Form the vector \(\vec{x_1}\) and determine the number of bicycles at the two locations the next day by finding \(\vec{x_2} = A\vec{x_1}\text{.}\)
      🔗
      🔗
    🔗
    🔗
  2. Suppose that one day there are 1050 bicycles at location \(B\) and 450 at location \(C\text{.}\) How many bicycles were there at each location the previous day?
    🔗
    🔗
  3. Suppose that there are 500 bicycles at location \(B\) and 500 at location \(C\) on Monday. How many bicycles are there at the two locations on Tuesday? on Wednesday? on Thursday?
    🔗
    🔗
🔗
🔗

13.

This problem is a continuation of the previous problem.
  1. Let us define vectors
    \begin{equation*} \vec{v_1} = \twovec{5}{2}, ~~~ \vec{v_2} = \twovec{-1}{1}\text{.} \end{equation*}
    Show that
    \begin{equation*} A\vec{v_1} = \vec{v_1}, ~~~A\vec{v_2} = 0.3\vec{v_2}\text{.} \end{equation*}
    🔗
    🔗
  2. Suppose that \(\vec{x_1} = c_1 \vec{v_1} + c_2 \vec{v_2}\) where \(c_1\) and \(c_2\) are scalars. Use the Linearity Principle expressed in Proposition 2.2.3 to explain why
    \begin{equation*} \vec{x_2} = A\vec{x_1} = c_1\vec{v_1} + 0.3c_2\vec{v_2}\text{.} \end{equation*}
    🔗
    🔗
  3. Continuing in this way, explain why
    \begin{equation*} \begin{aligned} \vec{x_3} = A\vec{x_2} \amp {}={} c_1\vec{v_1} +0.3^2c_2\vec{v_2} \\ \vec{x_4} = A\vec{x_3} \amp {}={} c_1\vec{v_1} +0.3^3c_2\vec{v_2} \\ \vec{x_5} = A\vec{x_4} \amp {}={} c_1\vec{v_1} +0.3^4c_2\vec{v_2} \\ \end{aligned}\text{.} \end{equation*}
    🔗
    🔗
  4. Suppose that there are initially 500 bicycles at location \(B\) and 500 at location \(C\text{.}\) Write the vector \(\vec{x_1}\) and find the scalars \(c_1\) and \(c_2\) such that \(\vec{x_1}=c_1\vec{v_1} + c_2\vec{v_2}\text{.}\)
    🔗
    🔗
  5. Use the previous part of this problem to determine \(\vec{x_2}\text{,}\) \(\vec{x_3}\) and \(\vec{x_4}\text{.}\)
    🔗
    🔗
  6. After a very long time, how are all the bicycles distributed?
    🔗
    🔗
🔗
🔗
🔗
🔗
PrevTopNext