 # MA211: Linear Algebra

Unit 3: Spectral Theory, Matrices, and Inner Product   *We have seen that matrix A corresponds to a linear transformation, T, i.e. T(x) =Ax.  If the matrix is square, then those nonzero vectors, any of which are transformed to a multiple of themselves, are called eigenvectors of the matrix, and the mutliples by which they are transformed are called eigenvalues (eigenis the German word for characteristic).  The set of eigenvalues is called the spectrum of A and plays an important role in linear algebra.  In this unit, you will study the spectrum of a square matrix in detail.

The first thing that we will see is that the eigenvalues of an nxn matrix correspond to the roots of an nth degree polynomial, called the characteristic polynomial of A.  As you know from algebra, roots of a polynomial can have multiplicity larger than 1.  For eigenvalues, the multiplicity of a root is called the algebraic multiplicity of the eigenvalue, while the dimension of the set of vectors for which it is an eigenvalue is called the geometric multiplicity of the eigenvalue.  If these two multiplicities are the same for all eigenvalues, then you will see that A is similar to a matrix with nonzero entries only along the main diagonal, that is A=S-1DS for an invertible matrix S.  In fact, the diagonal entries of D are the eigenvalues of A, and the columns of S are a basis of Rn consisting of eigenvectors.  Since solving polynomial equations can be difficult, you will study methods of estimating eigenvalues just by looking at the matrix.

You will then consider situations where the eigenvalues are known to be real and the matrix S, which diagonalizes A, can take a special form.  If the matrix A is symmetric, that is aij=aji  for all i,j, then you will see that A can be diagonalized, that is, all roots of the characteristic polynomial are real numbers.  Further, you will learn how to choose the basis of eigenvalues, which comprise S so that the rows and columns each are unit vectors and are mutually perpendicular.  Such matrices are called orthogonal and have the property that S-1=ST, where ST is the matrix obtained from S by interchanging its rows and columns.  Since not all square matrices are diagonalizable, you will need one of the most important theorems in the spectral theory of matrices: Schur’s Theorem, which is useful for analyzing the structure of matrices.

In linear algebra, you usually want to see whether or not Ax = b has any solutions.  Often Ax = b has no solution because there are more equations than unknowns, that is, the linear system is inconsistent and b is not in the column space of A.  In this case, you can try to find an approximation using a very important technique of the least square approximation.  Another thing you want to do is to characterize when Ax = b has a solution, that is, construct conditions for solvability of the system.  One way of doing this is by using Fredholm’s Alternative, which is discussed in this unit.  Fredholm’s Alternative is important, because it can be generalized to more general vector spaces, where the concept of rank of a determinant is not defined.  Finally, this unit discusses an important result known as the singular value decomposition, which gives a factorization of a matrix. *

Unit 3 Time Advisory
This unit should take you approximately 19 hours to complete.

***☐    ***Subunit 3.1: 8 hours

***☐    ***Sub-subunit 3.1.1: 2 hours

***☐    ***Sub-subunit 3.1.2: 1 hour

***☐    ***Sub-subunit 3.1.3: 5 hours

***☐    ***Reading: 0.5 hour

***☐    ***Assignment: 4.5 hours

***☐    ***Subunit 3.2: 11 hours

***☐    ***Sub-subunit 3.2.1: 2 hours

***☐    ***Sub-subunit 3.2.2: 2 hours

***☐    ***Sub-subunit 3.2.3: 1 hour

***☐    ***Sub-subunit 3.2.4: 6 hours

***☐    ***Reading: 2 hours

***☐    ***Assignment: 4 hours

Unit3 Learning Outcomes
Upon successful completion of this unit, you will be able to:
- find the algebraic multiplicity of an eigenvalue; - find the geometric multiplicity of an eigenvalue; - write and identify the characteristic equation for a matrix; - explain what it means for matrices to be similar; - determine whether a matrix is diagonalizable, and diagonalize matrices; - find the matrix exponential; - find the characteristic equation, eigenvalues, and corresponding eigenvectors of a given matrix; - estimate the eigenvalues for matrices; - state the eigenvalue problem from an algebraic perspective; - state the eigenvalue problem from a geometric perspective; - determine whether a given matrix is defective; - calculate the eigenvalues and corresponding eigenvectors for a linear transformation; - define orthogonal matrices; - determine whether a given matrix is orthogonal; - find eigenvalues of a symmetric and a skew-symmetric matrix; - find an orthonormal basis of eigenvectors for the matrix; - ­determine whether a given matrix is symmetric; - determine whether a given matrix is skew-symmetric; - define an orthonormal set of vectors; - use the Gram-Schmidt process to find an orthonormal basis; - find the least squares solution to a minimization problem; and - provide a factorization for a matrix using singular value decomposition.

3.1 Spectral Theory   3.1.1 Eigenvalues and Eigenvectors of a Matrix   - Reading: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 12: Spectral Theory:” “Section 12.1: Eigenvalues and Eigenvectors of a Matrix” Link: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 12: Spectral Theory:” “Section 12.1: Eigenvalues and Eigenvectors of a Matrix” (PDF)

Instructions: Please click on the link above, and read Section 12.1 on pages 215–231.  Spectral Theory refers to the study of eigenvalues and eigenvectors of a matrix, which is introduced in Section 12.1.  This reading should take you approximately 2 hours to complete.

3.1.2 The Estimation of Eigenvalues   - Reading: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 12: Spectral Theory:” “Section 12.3: The Estimation of Eigenvalues” Link: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 12: Spectral Theory:” “Section 12.3: The Estimation of Eigenvalues” (PDF)

Instructions: Please click on the link above and read Section 12.3 on pages 236–237.  Section 12.3 introduces Gerschgorin’s Theorem, which provides a way to estimate where the eigenvalues are just from looking at the matrix.  This reading should take you approximately 30 minutes to complete.

• Assessment: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 12: Spectral Theory:” “Section 12.4: Exercises” Link: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 12: Spectral Theory:” “Section 12.4: Exercises” (PDF)

Instructions: Please click on the above link to open the PDF.  Scroll down to page 237, and complete problems 3, 5, 8, 11, 13, 20, 25, 28, 43, 48, and 54.  Next, click on “Solutions” (PDF) and check your answers on pages 76–84.  This assessment should take you approximately 4 hours and 30 minutes to complete.

3.2 Matrices and Inner Product   3.2.1 Symmetric and Orthogonal Matrices   - Reading: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 13: Matrices and the Inner Product:” “Section 13.1: Symmetric and Orthogonal Matrices” Link: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 13: Matrices and the Inner Product:” “Section 13.1: Symmetric and Orthogonal Matrices” (PDF)

Instructions: Please click on the link above, and read Section 13.1 on pages 245–255.  Section 13.1 will introduce symmetric and orthogonal matrices.  This reading should take you approximately 2 hours to complete.

3.2.2 Fundamental Theory and Generalizations   - Reading: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 13: Matrices and the Inner Product:” “Section 13.2: Fundamental Theory and Generalizations” Link: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 13: Matrices and the Inner Product:” “Section 13.2: Fundamental Theory and Generalizations” (PDF)

Instructions: Please click on the link above, and read Section 13.2 on pages 255–262.  Sections 13.2 will discuss several results including the Gram-Schmidt process and the Schur’s Theorem.  This reading should take you approximately 2 hours to complete.

3.2.3 Least Square Approximation   - Reading: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 13: Matrices and the Inner Product:” “Section 13.3: Least Square Approximation” Link: Professor Kenneth Kuttler’s Elementary Linear Algebra: “Chapter 13: Matrices and the Inner Product:” “Section 13.3: Least Square Approximation” (PDF)

Instructions: Please click on the link above, and read Section 13.3 on pages 263–266.  Sections 13.3 discusses a very important technique known as the Least Square Approximation.  This reading should take you approximately 1 hour to complete.