MAT 350: Introduction to Eigenvectors and Eigenvalues

Warm-Up Problems

Complete the following warm-up problems to re-familiarize yourself with concepts we’ll be leveraging today.

Example: Consider the matrix $A = \begin{bmatrix} 7 & 6\\ 6 & -2\end{bmatrix}$. Evaluate the following matrix-vector products.

$A\begin{bmatrix} -1\\ 3\end{bmatrix}$
$A\begin{bmatrix} 2\\ 1\end{bmatrix}$

$A\begin{bmatrix} -1\\ 2\end{bmatrix}$
$A\begin{bmatrix} 5\\ 3\end{bmatrix}$

Look closely at the matrix-vector products and determine if there is anything particularly interesting about two of them.

Reminders and Today’s Goal

Multiplying an $n\times 1$ vector $\vec{x}$ by an $m\times n$ matrix transforms the vector from $\mathbb{R}^n$ to $\mathbb{R}^m$
- If the matrix is an $n\times n$ matrix, then the vector is moved within $\mathbb{R}^n$
- Matrix-vector multiplication with a square matrix max perform rotations, reflections, vertical shears, horizontal shears, stretches, compressions, or a combination of those transformatons.

Goals for Today: After today’s discussion, you should be able to

define what is meant by an eigenvector
describe what is meant by an eigenvalue
articulate the connection between an eigenvector and its eigenvalue
use properties of eigenvalues and eigenvectors to quickly perform matrix-vector multiplication

Motivating Utility of Eigenvectors and Eigenvalues

We’ll motivate why we might care about eigenvectors and eigenvalues even before defining them! We’ll use an example to do it.

Example: Consider the matrix $A = \begin{bmatrix} 7 & 6\\ 6 & -2\end{bmatrix}$ and the vectors $\vec{v_1} = \begin{bmatrix} 2\\ 1\end{bmatrix}$ and $\vec{v_2} = \begin{bmatrix} -1\\ 2\end{bmatrix}$ from the warm-up problems.

If you didn’t already do so, compute $A\vec{v_1}$.
Similarly, compute $A\vec{v_2}$ if you haven’t already.
Compute $A\left(5\vec{v_1}\right)$
Compute $A\left(\vec{v_1} + \vec{v_2}\right)$
Compute $A\left(3\vec{v_1} - 6\vec{v_2}\right)$

Motivating Utility of Eigenvectors and Eigenvalues

We’ll motivate why we might care about eigenvectors and eigenvalues even before defining them! We’ll use an example to do it.

Example: Consider the matrix $A = \begin{bmatrix} 7 & 6\\ 6 & -2\end{bmatrix}$ and the vectors $\vec{v_1} = \begin{bmatrix} 2\\ 1\end{bmatrix}$ and $\vec{v_2} = \begin{bmatrix} -1\\ 2\end{bmatrix}$ from the warm-up problems.

Compute $A^2\vec{v_1}$

Compute $A^5\vec{v_2}$
Notice that $\left\{\vec{v_1},~\vec{v_2}\right\}$ is a basis for $\mathbb{R}^2$ and that the vector $\vec{u} = \begin{bmatrix} 12\\ 11\end{bmatrix}$ can be written as $\vec{u} = 7\vec{v_1} + 2\vec{v_2}$. Compute $A\vec{u}$.

Eigenvalues and Eigenvectors

Definition (Eigenvector and Eigenvalue): An eigenvector of an $n\times n$ matrix $A$ is a non-zero vector $\vec{x}\in\mathbb{R}^n$ such that $A\vec{x} = \lambda\vec{x}$ for some scalar $\lambda$.

A scalar $\lambda$ is called an eigenvalue of $A$ if there is a non-trivial solution to the matrix equation $A\vec{x} = \lambda\vec{x}$.

Geometrical Note: An eigenvector of the matrix $A$ is a vector $\vec{x}$ whose transformed position after left-multiplication by the matrix $A$ is just a scaling of the vector from its original position.

These are vectors whose directionality has remained fixed (or have been rotated by $180^\circ$).

Identifying Eigenvectors and Eigenvalues

Strategy: Given a matrix $A$ and a non-zero vector $\vec{v}$, the vector is an eigenvector of $A$ if $A\vec{x} = \lambda\vec{x}$. That is, the matrix multiplication simply scales the original vector.

Example: Determine whether either of the vectors $\vec{v} = \begin{bmatrix} 1\\ 0\\ 1\end{bmatrix}$ or $\vec{u} = \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}$ is an eigenvector of the matrix $A = \begin{bmatrix} 4 & 1 & 0\\ 0 & 2 & 0\\ 1 & 0 & 3\end{bmatrix}$. If either is an eigenvector, identify its corresponding eigenvalue.

Solution Path. To answer this, do out the multiplication $A\vec{v}$ and $A\vec{u}$. Notice that $A\vec{v} = 4\vec{v}$, but $A\vec{u}$ is not simply a scaled copy of $\vec{u}$. The eigenvalue corresponding to the eigenvector $\vec{v}$ is $\lambda = 4$.

Finding Eigenvectors and Eigenvalues

This will be the focus of our next discussion, but for now, a preview…

Strategy (Finding Eigenvectors and Eigenvalues): Recall that a scalar $\lambda$ is an eigenvalue of the matrix $A$ if there exists a non-trivial (not $\vec{0}$) solution to the matrix equation $A\vec{x} = \lambda \vec{x}$.

Intuition: We haven’t solved equations with unknowns on both sides of an equal sign this semester, so we’ll begin by rewriting the equation

\[\begin{align*} A\vec{x} &= \lambda\vec{x} \end{align*}\]

Finding Eigenvectors and Eigenvalues

This will be the focus of our next discussion, but for now, a preview…

Strategy (Finding Eigenvectors and Eigenvalues): Recall that a scalar $\lambda$ is an eigenvalue of the matrix $A$ if there exists a non-trivial (not $\vec{0}$) solution to the matrix equation $A\vec{x} = \lambda \vec{x}$.

Intuition: We haven’t solved equations with unknowns on both sides of an equal sign this semester, so we’ll begin by rewriting the equation

\[\begin{align*} A\vec{x} &= \lambda\vec{x}\\ \implies A\vec{x} - \lambda\vec{x} &= \vec{0} \end{align*}\]

Finding Eigenvectors and Eigenvalues

This will be the focus of our next discussion, but for now, a preview…

Strategy (Finding Eigenvectors and Eigenvalues): Recall that a scalar $\lambda$ is an eigenvalue of the matrix $A$ if there exists a non-trivial (not $\vec{0}$) solution to the matrix equation $A\vec{x} = \lambda \vec{x}$.

Intuition: We haven’t solved equations with unknowns on both sides of an equal sign this semester, so we’ll begin by rewriting the equation

\[\begin{align*} A\vec{x} &= \lambda\vec{x}\\ \implies A\vec{x} - \lambda\vec{x} &= \vec{0}\\ \implies \left(A - \lambda I\right)\vec{x} &= \vec{0} \end{align*}\]

The bottom equation will have a unique solution if $\left(A - \lambda I\right)$ is invertible.

We’ll have non-trivial solutions if $\left(A - \lambda I\right)$ is not invertible.

Finding Eigenvectors and Eigenvalues

Strategy (Finding Eigenvectors and Eigenvalues): Recall that a scalar $\lambda$ is an eigenvalue of the matrix $A$ if there exists a non-trivial (not $\vec{0}$) solution to the matrix equation $A\vec{x} = \lambda \vec{x}$.

Note: The following items follow directly from what we just saw and are worth noting regarding eigenvalues.

Notice that $\lambda$ is an eigenvalue of $A$ if $\left(A - I_n\right)\vec{x} = \vec{0}$ has a non-trivial solution.
Notice that $\lambda$ is an eigenvalue of $A$ if $\begin{bmatrix} A - \lambda I_n & | & \vec{0}\end{bmatrix}$ has a free variable.
Notice that $\lambda$ is an eigenvalue of $A$ if the matrix $A - \lambda I_n$ is not invertible.
Notice that $\lambda$ is an eigenvalue of $A$ if $\text{dim}\left(\text{Nul}\left(A - \lambda I_n\right)\right) > 0$.
Notice that, if $\lambda$ is an eigenvalue of $A$, then $\text{Nul}\left(A - \lambda I_n\right)$ is a subspace of $\mathbb{R}^n$ corresponding to the eigenvalue $\lambda$. This subspace is often referred to as the eigenspace of $A$ corresponding to $\lambda$.

Completed Example #2

Example: Determine whether $\lambda = 5$ is an eigenvalue for the matrix $A = \left[\begin{array}{rr} 6 & 8\\ 1 & 13\end{array}\right]$. If it is an eigenvalue, find an eigenvector corresponding to $\lambda = 5$.

We’ll start by solving the matrix equation $\left(A - 5I\right)\vec{x} = \vec{0}$.

As usual, we’ll do this by constructing a corresponding augmented matrix and row-reducing.

\[\left[\begin{array}{rr|r} 6 - 5 & 8 & 0\\ 1 & 13 - 5 & 0\end{array}\right]\]

import sympy as sp

A = sp.Matrix([[6 - 5, 8, 0], [1, 13 - 5, 0]])
A.rref()

(Matrix([
[1, 8, 0],
[0, 0, 0]]), (0,))

There is a free variable here, so $\lambda = 5$ is indeed a eigenvalue for this matrix.

Completed Example #2

Example: Determine whether $\lambda = 5$ is an eigenvalue for the matrix $A = \left[\begin{array}{rr} 6 & 8\\ 1 & 13\end{array}\right]$. If it is an eigenvalue, find an eigenvector corresponding to $\lambda = 5$.

We’ll start by solving the matrix equation $\left(A - 5I\right)\vec{x} = \vec{0}$.

As usual, we’ll do this by constructing a corresponding augmented matrix and row-reducing.

import sympy as sp

A = sp.Matrix([[6 - 5, 8, 0], [1, 13 - 5, 0]])
A.rref()

(Matrix([
[1, 8, 0],
[0, 0, 0]]), (0,))

There is a free variable here, so $\lambda = 5$ is indeed a eigenvalue for this matrix.

We can construct a basis for the eigenspace of this matrix corresponding to $\lambda = 5$ by writing the solutions to the equation we began from, in parameteric vector form. Note that $\vec{x} = x_2\left[\begin{array}{r} -8\\ 1\end{array}\right]$. Thus, $\mathscr{B}_{\lambda = 5} = \left\{\left[\begin{array}{r} -8\\ 1\end{array}\right]\right\}$.

Eigenvalues of Triangular Matrices

Recall (Triangular Matrix): A matrix $A$ having all entries either above or below its main diagonal as $0$’s is called a triangular matrix. If the $0$’s are below the main diagonal, $A$ is called lower triangular while a matrix having all $0$’s above the main diagonal is $upper triangular*.

Theorem (Eigenvalues of Triangular Matrices): The eigenvalues of a triangular matrix are the entries along its main diagonal.

Example: The eigenvalues of the matrix $A = \begin{bmatrix} 6 & 0 & 0 & 0\\ -2 & 0 & 0 & 0\\ -1 & 3 & 1 & 0\\ 2 & 0 & -7 & 5\end{bmatrix}$ are…

Eigenvalues of Triangular Matrices

Recall (Triangular Matrix): A matrix $A$ having all entries either above or below its main diagonal as $0$’s is called a triangular matrix. If the $0$’s are below the main diagonal, $A$ is called lower triangular while a matrix having all $0$’s above the main diagonal is $upper triangular*.

Theorem (Eigenvalues of Triangular Matrices): The eigenvalues of a triangular matrix are the entries along its main diagonal.

Example: The eigenvalues of the matrix $A = \begin{bmatrix} 6 & 0 & 0 & 0\\ -2 & 0 & 0 & 0\\ -1 & 3 & 1 & 0\\ 2 & 0 & -7 & 5\end{bmatrix}$ are $\lambda_1 = 6$, $\lambda_2 = 0$, $\lambda_3 = 1$, and $\lambda_4 = 5$.

Theorem: If $\vec{v_1}, \vec{v_2}, \cdots, \vec{v_r}$ are eigenvectors that correspond to distinct eigenvalues $\lambda_1, \lambda_2, \cdots, \lambda_r$ of an $n\times n$ matrix $A$ then the set $\left\{\vec{v_1}, \vec{v_2}, \cdots, \vec{v_r}\right\}$ are linearly independent.

Eigenvectors and Eigenvalues, Why Care?

Scalar multiplication is much simpler and faster than matrix multiplication.
Eigenvectors allow us to replace matrix multiplication by scalar multiplication.
Recall the superposition principle of linear transformations which is often utilized in physics and engineering

\[T\left(c_1\vec{v_1} + c_2\vec{v_2} + \cdots + c_n\vec{v_n}\right) = c_1T\left(\vec{v_1}\right) + c_2T\left(\vec{v_2}\right) + \cdots + c_nT\left(\vec{v_n}\right)\]

This is particularly useful when $\left\{\vec{v_1}, \vec{v_2}, \cdots, \vec{v_n}\right\}$ form a basis for $\mathbb{R}^n$.
Even better, if we had a basis consisting of eigenvectors (an eigenbasis), then there is no need for matrix multiplication to be carried out at all.
- Write a generic vector $\vec{x}$ as a linear combination of the eigenbasis vectors and then evaluate $T\left(\vec{x}\right)$ using scalar multiplication and addition.

Eigenvectors and Eigenvalues, Why Care?

Additionally, eigenvectors and eigenvalues indicate how much (eigenvalues) and in what directions (eigenvectors) a linear transformation stretches space.
We’ve seen that changing bases from the standard basis to an alternative basis (like an eigenbasis) can be helpful.
If one or more of the eigenvectors (axes in the eigenbasis representation of space) have very small eigenvalues, then…
- those extra spatial dimensions may not be necessary
- we may be able to drop them without much information loss

Examples to Try

Example 1: Determine whether $\lambda = 3$ is an eigenvalue for the matrix $A = \left[\begin{array}{rr} 5 & 6\\ -2 & 4\end{array}\right]$. If it is an eigenvalue, find a corresponding eigenvector.

Example 2: Determine whether the vector $\left[\begin{array}{r} -5\\ -4\\ 3\end{array}\right]$ is an eigenvector for the matrix $A = \left[\begin{array}{rrr} 0 & 5 & -10\\ 0 & 22 & 16\\ 0 & -9 & -2\end{array}\right]$. If so, find the corresponding eigenvalue and at least one other eigenvector corresponding to the same eigenvalue.

Examples to Try (2 of 2)

Example 3: The matrix $A = \left[\begin{array}{rr} 8 & 2\\ 6 & 12\end{array}\right]$ has eigenvalues $\lambda = 6$ and $\lambda = 14$. Find a basis for each of the corresponding eigenspaces.

Example 4: Find the eigenvalues corresponding to the matrix $A = \left[\begin{array}{rr} 1 & 0 & 0\\ 2 & 2 & 0\\ 1 & 0 & 5\end{array}\right]$.

Homework

\[\Huge{\text{Start Homework 11}}\] \[\Huge{\text{on MyOpenMath}}\]

Next Time…

$\Huge{\text{Finding}}$

$\Huge{\text{Eigenvectors and Eigenvalues}}$