August 8, 2025
Complete the following warm-up problems to re-familiarize yourself with matrix multiplication.
Compute the matrix-vector product \(\begin{bmatrix} 1 & 0 & -3\\ 2 & 4 & -2\\ 0 & 0 & 5\end{bmatrix}\begin{bmatrix} 3\\ -2\\ 1\end{bmatrix}\)
Compute the matrix-vector product \(\begin{bmatrix} 1 & 0 & -3\\ 2 & 4 & -2\\ 0 & 0 & 5\\ -1 & -1 & 0\end{bmatrix}\begin{bmatrix} 3\\ -2\\ 1\end{bmatrix}\)
Compute the matrix product \(\begin{bmatrix} 2 & -1 & 0 & 6\\ 1 & 1 & -1 & 1\\ 0 & 4 & 1 & 0\end{bmatrix}\begin{bmatrix} 2 & -1\\ 0 & 8\\ 1 & 1\\ 3 & -1\end{bmatrix}\)
The vector \(\vec{y} = c_1\vec{v_1} + c_2\vec{v_2} + \cdots + c_k\vec{v_k}\) is a linear combination of the vectors \(\vec{v_1},~\vec{v_2},~\cdots,~\vec{v_k}\) with weights \(c_1,~c_2,~\cdots,~c_k\)
We can view the matrix \(A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & \cdots & a_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ a_{m1} & a_{m2} & \cdots & a_{mn}\end{bmatrix}\) as a collection of column vectors, \(A = \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & a_{n}\end{bmatrix}\)
The number of columns in a matrix \(A\) must match the number of entries in a column vector \(\vec{v}\) in order for the matrix-vector product \(A\vec{v}\) to be defined.
Goal for Today: We revisit linear combinations, but rather than weighted sums of vectors, we consider how linear combinations can manifest themselves within matrix-vector products.
Consider the matrix product \(A\vec{x}\), where \(A = \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\)
\[\begin{align} &\begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n\end{bmatrix} \end{align}\]
Consider the matrix-vector product
\[\begin{align} &\begin{bmatrix} 1 & 5 & -3\\ 8 & 4 & 0\\ -2 & 2 & 1\\ 0 & 0 & -1\end{bmatrix}\begin{bmatrix} 3\\ 2\\ -4\end{bmatrix} \end{align}\]
Consider the matrix product \(A\vec{x}\), where \(A = \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\)
\[\begin{align} &\begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n\end{bmatrix} \end{align}\]
Consider the matrix-vector product
\[\begin{align} &\begin{bmatrix} 1 & 5 & -3\\ 8 & 4 & 0\\ -2 & 2 & 1\\ 0 & 0 & -1\end{bmatrix}\begin{bmatrix} 3\\ 2\\ -4\end{bmatrix}\\ &= \begin{bmatrix} 1\left(3\right) + 5\left(2\right) + \left(-3\right)\left(-4\right)\\ 8\left(3\right) + 4\left(2\right) + 0\left(-4\right)\\ -2\left(3\right) + 2\left(2\right) + 1\left(-4\right)\\ 0\left(3\right) + 0\left(2\right) + \left(-1\right)\left(-4\right)\end{bmatrix} \end{align}\]
Consider the matrix product \(A\vec{x}\), where \(A = \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\)
\[\begin{align}\begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n\end{bmatrix} \end{align}\]
Consider the matrix-vector product
\[\begin{align} &\begin{bmatrix} 1 & 5 & -3\\ 8 & 4 & 0\\ -2 & 2 & 1\\ 0 & 0 & -1\end{bmatrix}\begin{bmatrix} 3\\ 2\\ -4\end{bmatrix}\\ &= 3\begin{bmatrix} 1\\ 8\\ -2\\ 0\end{bmatrix} + 2\begin{bmatrix} 5\\ 4\\ 2\\ 0\end{bmatrix} + \left(-4\right)\begin{bmatrix} -3\\ 0\\ 1\\ -1\end{bmatrix} \end{align}\]
Consider the matrix product \(A\vec{x}\), where \(A = \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\)
\[\begin{align} &\begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n\end{bmatrix}\\ &= x_1\vec{a_1} + x_2\vec{a_2} + \cdots + x_n\vec{a_n} \end{align}\]
Consider the matrix-vector product
\[\begin{align} &\begin{bmatrix} 1 & 5 & -3\\ 8 & 4 & 0\\ -2 & 2 & 1\\ 0 & 0 & -1\end{bmatrix}\begin{bmatrix} 3\\ 2\\ -4\end{bmatrix}\\ &= 3\begin{bmatrix} 1\\ 8\\ -2\\ 0\end{bmatrix} + 2\begin{bmatrix} 5\\ 4\\ 2\\ 0\end{bmatrix} + \left(-4\right)\begin{bmatrix} -3\\ 0\\ 1\\ -1\end{bmatrix} \end{align}\]
Consider the matrix product \(A\vec{x}\), where \(A = \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\)
\[\begin{align} &\begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n\end{bmatrix}\\ &= \boxed{~x_1\vec{a_1} + x_2\vec{a_2} + \cdots + x_n\vec{a_n}~} \end{align}\]
A linear combination of the columns of A
Consider the matrix-vector product
\[\begin{align} &\begin{bmatrix} 1 & 5 & -3\\ 8 & 4 & 0\\ -2 & 2 & 1\\ 0 & 0 & -1\end{bmatrix}\begin{bmatrix} 3\\ 2\\ -4\end{bmatrix}\\ &= 3\begin{bmatrix} 1\\ 8\\ -2\\ 0\end{bmatrix} + 2\begin{bmatrix} 5\\ 4\\ 2\\ 0\end{bmatrix} + \left(-4\right)\begin{bmatrix} -3\\ 0\\ 1\\ -1\end{bmatrix} \end{align}\]
We’ve already mentioned this equivalency, what we’ve just noted about the connection between matrix-vector products and linear combinations of the columns of \(A\) is what allows us to claim matrix equations and vector equations are equivalent.
\[\begin{align} A\vec{x} &= \vec{b} \end{align}\]
We’ve already mentioned this equivalency, what we’ve just noted about the connection between matrix-vector products and linear combinations of the columns of \(A\) is what allows us to claim matrix equations and vector equations are equivalent.
\[\begin{align} A\vec{x} &= \vec{b}\\ \implies \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n\end{bmatrix} &= \vec{b} \end{align}\]
We’ve already mentioned this equivalency, what we’ve just noted about the connection between matrix-vector products and linear combinations of the columns of \(A\) is what allows us to claim matrix equations and vector equations are equivalent.
\[\begin{align} A\vec{x} &= \vec{b}\\ \implies \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n\end{bmatrix} &= \vec{b}\\ \implies x_1\vec{a_1} + x_2\vec{a_2} + \cdots + x_n\vec{a_n} &= \vec{b} \end{align}\]
While a matrix equation is indeed equivalent to a vector equation, the question a matrix equation naturally asks is not one about linear combinations.
Instead, we can (and will) think of matrix multiplication as a function which transforms an input vector into an output vector.
As we mentioned in our most recent discussion, we can think of a function \(f: \mathbb{R}^n \to \mathbb{R}^m\) defined by \(f\left(\vec{x}\right) = A\vec{x}\).
Example: Solve the matrix equation \(A\vec{x} = \vec{b}\) where \(A = \begin{bmatrix} 1 & -2 & 0 & 3\\ 4 & 1 & 5 & -1\end{bmatrix}\) and \(\vec{b} = \begin{bmatrix} 7\\ 20\end{bmatrix}\)
While not generally the way we multiply matrices, pointing out that a matrix can be thought of as a collection of column vectors allows us to perform matrix-matrix multiplication as a series of matrix-vector products.
\[\begin{align} AB &= A\begin{bmatrix} \vec{b_1} & \vec{b_2} & \cdots & \vec{b_k}\end{bmatrix}\\ &= \begin{bmatrix} A\vec{b_1} & A \vec{b_2} & \cdots & A\vec{b_k}\end{bmatrix} \end{align}\]
That is, the \(i^{\text{th}}\) column of the matrix \(AB\) is \(A\vec{b_i}\), the product of the matrix \(A\) and the \(i^{\text{th}}\) column of the matrix \(B\).
We won’t generally make use of this, but it is a fact worth observing.
We’ve seen that we can view a matrix \(A\) as a collection of column vectors, but we could also view the matrix as a collection of row vectors.
Consider the matrix \(A = \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\) to be a collection of column vectors and the matrix \(B = \begin{bmatrix} \leftarrow \vec{b_{1}^{*}} \rightarrow\\ \leftarrow \vec{b_{2}^{*}} \rightarrow\\ \vdots\\ \leftarrow \vec{b_{m}^{*}} \rightarrow \end{bmatrix}\) as a collection of row-vectors.
Now the matrix-matrix product \(AB\) can be viewed as
\[\begin{align} AB &= \begin{bmatrix} \vec{a_1} & \vec{a_2} & \cdots & \vec{a_n}\end{bmatrix}\begin{bmatrix} \leftarrow \vec{b_{1}^{*}} \rightarrow\\ \leftarrow \vec{b_{2}^{*}} \rightarrow\\ \vdots\\ \leftarrow \vec{b_{m}^{*}} \rightarrow \end{bmatrix}\\ &= \vec{a_1}\vec{b_{1}^*} + \vec{a_2}\vec{b_{2}^*} + \cdots + \vec{a_n}\vec{b_{m}^*} \end{align}\]
A decomposition we won’t use in MAT350, but which is critical to data compression, dimension reduction, and advanced techniques in machine learning.
Example: Let \(A = \begin{bmatrix} 1 & 0 & -2\\ 2 & 1 & 1\\ 0 & 3 & -1\end{bmatrix}\) and \(\vec{b} = \begin{bmatrix} -1\\ 7\\ 4\end{bmatrix}\). Solve the matrix equation \(A\vec{x} = \vec{b}\) and identify the solution vector using parametric vector form if necessary.
Example: Find all of the solutions to the matrix equation \(A\vec{x} = \vec{b}\) if \(A = \begin{bmatrix} 1 & 2 & -1\\ 2 & 4 & -2\end{bmatrix}\) and \(\vec{b} = \begin{bmatrix} 3\\ 7\end{bmatrix}\).
Example: Solve the matrix equation \(\begin{bmatrix} 1 & -2 & 1\\ 0 & 0 & 0\end{bmatrix}\vec{x} = \begin{bmatrix} 4\\ 0\end{bmatrix}\), express the solution in parametric vector form, and describe the geometry of the solution space.
Example: Determine whether the vector \(\vec{b} = \left[\begin{array}{c} 0\\ 5\\ -1\end{array}\right]\) is a linear combination of the columns of the matrix \(A = \left[\begin{array}{rrr} 1 & 2 & -1\\ 3 & 0 & 2\\ -1 & 1 & 0\end{array}\right]\).
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
To keep track of the bicycles, we form a vector \(\vec{x_k} = \begin{bmatrix} D_k\\ P_k\end{bmatrix}\) where \(D_k\) is the number of bicycles at location \(D\) and \(P_k\) is the number of bicycles at location \(p\) at the beginning of day \(k\).
The information gathered tells us that we can construct the system
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
The information gathered tells us that we can construct the system (left) with corresponding matrix form (right)
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
\[\vec{x_{k+1}} = \begin{bmatrix} 0.8 & 0.5\\ 0.5 & 0.5\end{bmatrix} \vec{x_k}\]
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
The information gathered tells us that we can construct the system (left) with corresponding matrix form (right)
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
\[\vec{x_{k+1}} = \begin{bmatrix} 0.8 & 0.5\\ 0.5 & 0.5\end{bmatrix} \vec{x_k}\]
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
The information gathered tells us that we can construct the system (left) with corresponding matrix form (right)
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
\[\vec{x_{k+1}} = \begin{bmatrix} 0.8 & 0.5\\ 0.5 & 0.5\end{bmatrix} \vec{x_k}\]
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
The information gathered tells us that we can construct the system (left) with corresponding matrix form (right)
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
\[\vec{x_{k+1}} = \begin{bmatrix} 0.8 & 0.5\\ 0.5 & 0.5\end{bmatrix} \vec{x_k}\]
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
The information gathered tells us that we can construct the system (left) with corresponding matrix form (right)
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
\[\vec{x_{k+1}} = \begin{bmatrix} 0.8 & 0.5\\ 0.5 & 0.5\end{bmatrix} \vec{x_k}\]
\[\vec{x_2} = A\vec{x_1} = c_1\vec{v_1} + 0.3c_2\vec{v_2}\]
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
The information gathered tells us that we can construct the system (left) with corresponding matrix form (right)
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
\[\vec{x_{k+1}} = \begin{bmatrix} 0.8 & 0.5\\ 0.5 & 0.5\end{bmatrix} \vec{x_k}\]
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
The information gathered tells us that we can construct the system (left) with corresponding matrix form (right)
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
\[\vec{x_{k+1}} = \begin{bmatrix} 0.8 & 0.5\\ 0.5 & 0.5\end{bmatrix} \vec{x_k}\]
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
The information gathered tells us that we can construct the system (left) with corresponding matrix form (right)
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
\[\vec{x_{k+1}} = \begin{bmatrix} 0.8 & 0.5\\ 0.5 & 0.5\end{bmatrix} \vec{x_k}\]
Bike Share: Suppose that a city is starting a bicycle sharing program with bicycles at two locations – downtown \(D\) and the city park \(P\). Bicycles that are rented at one location may be returned to either location at the end of the day. Over time, the city finds that 80% of bicycles rented at location \(D\) are returned to \(D\) with the other 20% returned to \(P\). Similarly, 50% of bicycles rented at location \(P\) are returned to \(P\) and 50% to \(D\).
The information gathered tells us that we can construct the system (left) with corresponding matrix form (right)
\[\left\{\begin{array}{rcr} D_{k+1} &= 0.8D_k + 0.5P_k\\ P_{k+1} &= 0.5D_k + 0.5P_k\end{array}\right.\]
\[\vec{x_{k+1}} = \begin{bmatrix} 0.8 & 0.5\\ 0.5 & 0.5\end{bmatrix} \vec{x_k}\]
To be added…
\[\Huge{\text{Finish Homework 4}}\] \[\Huge{\text{on MyOpenMath}}\]
\(\Huge{\text{Spans of Sets of Vectors}}\)