Properties of Matrix Operations

Properties of Matrix Addition

The fundamental properties of real number addition also apply to matrices.

Let \( A \), \( B \), and \( C \) be \( m \times n \) matrices:

1. Commutative Property: \( A + B = B + A \)
2. Associative Property: \( A + (B + C) = (A + B) + C \)
3. Additive Identity: There exists a unique \( m \times n \) zero matrix \( O \) such that \( A + O = A \)
4. Additive Inverse: For every matrix \( A \), there exists \( -A \) such that \( A + (-A) = O \)

Properties of Matrix Multiplication

Unlike addition, not all multiplication properties of real numbers apply to matrices.

• Matrix multiplication is not commutative: Even if both \( AB \) and \( BA \) are defined, they may not be equal.
• A matrix may not have a multiplicative inverse, even if it is square.

However, some properties do generalize. Let \( A \), \( B \), and \( C \) be matrices such that the operations are defined:

1. Associative Property: \( A(BC) = (AB)C \)
2. Left Distributive Property: \( A(B + C) = AB + AC \)
3. Right Distributive Property: \( (A + B)C = AC + BC \)
4. Multiplicative Identity: \( I_m A = A \), \( A I_n = A \)

Properties of Scalar Multiplication

Let \( r \), \( s \) be real numbers, and \( A \), \( B \) be matrices:

1. \( r(sA) = (rs)A \)
2. \( (r + s)A = rA + sA \)
3. \( r(A + B) = rA + rB \)
4. \( A(rB) = r(AB) = (rA)B \) (if defined)

Properties of the Transpose of a Matrix

Let \( r \) be real, and \( A \), \( B \) be matrices:

1. \( (A^T)^T = A \)
2. \( (A + B)^T = A^T + B^T \)
3. \( (AB)^T = B^T A^T \)
4. \( (rA)^T = rA^T \)

Properties of Determinants

1. \( \det(A) = \det(A^T) \)
2. If any row/column is multiplied by \( k \), then \( \det(\Delta') = k \cdot \det(\Delta) \)
3. If a row or column is all zeros, then \( \det = 0 \)
4. If matrix is upper/lower triangular, \( \det = \) product of diagonal elements

Matrix Theory: Minor, Cofactor, Adjoint, and Inverse

Matrix Multiplication Representation

Let matrix \( A \) be of size \( i \times k \), and matrix \( B \) be of size \( k \times j \). Their product \( C = A \cdot B \) will result in a matrix of size \( i \times j \).

The element in the \( i^{\text{th}} \) row and \( j^{\text{th}} \) column of the resulting matrix \( C \), denoted by \( c_{ij} \), is calculated as:

\( c_{ij} = \sum_k a_{ik} \cdot b_{kj} \)

The transpose of matrix \( C \), denoted as \( C^T \), swaps rows and columns, resulting in a matrix of size \( j \times i \):

\( C^T = [C]_{j \times i} \), where \( C^T_{ji} = C_{ij} \)

1. Transpose of a Product

1.a. Proof that \( (AB)^T = B^T A^T \)

The transpose of a product of two matrices equals the product of their transposes in reverse order. That is:

\( (AB)^T = B^T A^T \)

To see why this is true, consider the element at position \( (i, j) \) in \( (AB)^T \):

\( (AB)^T_{ij} = (AB)_{ji} = \sum_k a_{jk} \cdot b_{ki} \)

On the other hand, consider the element \( (i, j) \) in \( B^T A^T \):

\( (B^T A^T)_{ij} = \sum_k b^T_{ik} \cdot a^T_{kj} = \sum_k b_{ki} \cdot a_{jk} \)

Since both expressions are the same, we conclude:

\( (AB)^T = B^T A^T \)

1.b. Reverse Direction \( B^T A^T = (AB)^T \)

We can also prove this identity starting from the right-hand side:

\( (B^T A^T)_{ij} = \sum_k b^T_{ik} \cdot a^T_{kj} = \sum_k b_{ki} \cdot a_{jk} = (AB)_{ji} = (AB)^T_{ij} \)

2. Cofactor and Adjoint

Cofactor Matrix

The cofactor of the element in the \( i^{\text{th}} \) row and \( j^{\text{th}} \) column of a square matrix \( A \) is given by:

\( \text{CO}_{ij} = (-1)^{i+j} \cdot M_{ij} \)

Where \( M_{ij} \) is the minor of the element, i.e., the determinant of the submatrix formed by deleting the \( i^{\text{th}} \) row and \( j^{\text{th}} \) column from \( A \).

Adjoint

The adjoint (or adjugate) of a square matrix \( A \) is the transpose of the matrix of cofactors:

\( \text{Adj}(A) = [\text{CO}_{ij}]^T \)

3. Inverse of a Matrix

If matrix \( A \) is invertible (i.e., \( \det(A) \ne 0 \)), then the inverse of \( A \) is given by:

\( A^{-1} = \frac{\text{Adj}(A)}{\det(A)} \)

4. Inverse of a Product

The inverse of a product of two invertible matrices is the product of their inverses in reverse order:

\( (AB)^{-1} = B^{-1} A^{-1} \)

Proof:

Start with the identity:

\( (AB)(AB)^{-1} = I \)

Pre-multiply both sides by \( A^{-1} \):

\( A^{-1}(AB)(AB)^{-1} = A^{-1} \Rightarrow B(AB)^{-1} = A^{-1} \)

Now pre-multiply both sides by \( B^{-1} \):

\( B^{-1}B(AB)^{-1} = B^{-1}A^{-1} \Rightarrow (AB)^{-1} = B^{-1}A^{-1} \)

5. Adjoint of a Product

The adjoint of the product of two matrices equals the product of their adjoints in reverse order:

\( \text{Adj}(AB) = \text{Adj}(B) \cdot \text{Adj}(A) \)

Why this works:

From the inverse formula:

\( (AB)^{-1} = \frac{\text{Adj}(AB)}{\det(AB)} \)

We also have:

\( (AB)^{-1} = B^{-1} A^{-1} \), and \( \det(AB) = \det(A) \cdot \det(B) \)

Substitute the inverse formulas:

\( A^{-1} = \frac{\text{Adj}(A)}{\det(A)} \), and \( B^{-1} = \frac{\text{Adj}(B)}{\det(B)} \)

Multiply the adjoints accordingly:

\( \text{Adj}(B) \cdot \text{Adj}(A) = \det(A) \cdot \det(B) \cdot B^{-1} \cdot A^{-1} \)

Thus, from the original inverse formula:

\( \text{Adj}(AB) = \text{Adj}(B) \cdot \text{Adj}(A) \)

Eigenvalue and Eigenvector

Let’s assume a square matrix A

The characteristic equation,

| A – λ*I | = 0  

(where I is an identity matrix)

After calculating the values of λs we attempt to find eigenvectors for corresponding eigenvalues like this

For eigenvalue, λ = λ₁

A*x = λ₁*I*x   (where, x is an unknown vector)

Or, (A - λ₁*I)*x = 0

The value of x is the corresponding eigenvector of λ₁

 

Power Method for Dominant Eigenvalue

Let λ₁, λ₂, λ₃, and λ_n be the eigenvalues of an n X n matrix A.  λ₁ is called the dominant eigenvalue of A if

| λ₁| > | λ_i |,   i = 2, 3, ... , n

The eigenvectors corresponding to λ₁ are called dominant eigenvectors of A.

 

Example

A =

 

We begin with an initial nonzero approximation of

x₀ =

 

We then obtain the following approximations

x₁ = Ax₀ = = = -4

 

x₂ = Ax₁ = = = 10

x₃ = Ax₂ = = = -22

 

x₄ = Ax₃ = = = 46

 

x₅ = Ax₄ = = = -94

x₆ = Ax₅ = = = 190

 

Note that the approximations in Example appear to be approaching scalar multiples of

So, the obtained dominant eigenvector from the above iterations is

x =

 

Now, we’ll find the corresponding eigenvalue from the obtained eigenvector

 

Formula

If x is an eigenvector of A, then its corresponding eigenvalue is given by

 

λ = (Ax.x / x.x)

Ax = =

Then, Ax.x = = -20.0 (approx.)

And x.x = = = 9.94 (approx.)

So, the corresponding eigenvalue, λ = (-20.0 / 9.94) = -2 (approx.)

 

Singular Value Decomposition (SVD)

Theory:

Singular Value Decomposition (SVD) is a matrix factorization technique that decomposes any m×n matrix A into three matrices: A=UΣV^T

Where:

• U is an m×m orthogonal matrix (or unitary if complex).
• Σ is m×n diagonal matrix with non-negative real numbers on the diagonal (singular values).
• V^T is an n×n orthogonal matrix (or unitary if complex), and V^T is the transpose of V.

Example

A =

 

Compute A^TA and AA^T

 

A^TA = =

 

AA^T = =

 

Find Eigenvalues and Eigenvectors:

For A^TA:

• Eigenvalues are λ1= 29.8661 and λ2 = 0.1339
• Corresponding eigenvectors (normalized) are v1 = and v2 =

For AA^T:

• Eigenvalues are λ1= 29.8661 and λ2 = 0.1339
• Corresponding eigenvectors (normalized) are u1 = and u2 =

 

Compute Singular Values:

Singular values are σ1 = √ 29.8661= 5.4650

And σ1 = √ 0.1339 = 0.3660

 

Final SVD:

A=UΣV^T

Where:

U =   Σ =  

V^T =

 

LU Decomposition

LU Decomposition is a method to find solutions of linear equations.

 

Using Gauss Elimination Method

 

Consider a matrix 𝐴. If all entries below the diagonal entries are zero, then the matrix is called “upper triangular.” If all entries above the diagonal entries are zero, then the matrix is called “lower triangular.”

         And A = L*U

L =  ;   U =

 

L= lower triangular matrix; U= upper triangular matrix

 

5. For a given matrix

A =

 

A=L*U

Or, A=*

 

L= lower triangular matrix; U= upper triangular matrix

 

After doing the row operation "R2 - (-2)*R1," we get,

 

Or, A = *

 

After row operation in matrix U, we've set (-2) to the same place of the L matrix and the (2, 1)^th position of the U matrix, which is now zero.

Firstly, try the first column elements of matrix U below diagonal elements to make zeroes,

 

After doing,          ‘R2-(-2)*R1’ (as demonstrated above)

                              ‘R3-(3)*R1’ and

                              ‘R4-(2)*R1’

We get,

 

A = *

 

In a same way, we will now employ row operations to set the elements of the second column of matrix U to zero.

 

Calculate,    ‘R3-(-4)*R2’ and

                    ‘R4-(1)*R2’

 

We get,

A = *

 

Now, we'll apply row operations to convert the elements of the third column of matrix U to zeroes.

 

Now calculate "R4-(3)*R2"

We get,

 

A = *

 

So,

 

L =

 

U =

 

6. Before showing the final result, all intermediate steps must be displayed.

 

Row Echelon Form

A matrix is in row echelon form if

• All rows consisting of only zeroes are at the bottom.
• The leading entry (that is the left-most nonzero entry) of every nonzero row is to the right of the leading entry of every row above
• Some texts add the condition that the leading coefficient must be 1 while others regard this as reduced row echelon form
• These two conditions imply that all entries in a column below a leading coefficient are zeros

 

 

Example

Given matrix,

A =

 

R2 ← R2 – R1

 

R3 ← R3 + 2*R2

 

R3 ← R3 / 4

 

Rank of a Matrix

Theory:

Definition: The rank of a matrix is defined as the maximum number of linearly independent rows (or columns) in the matrix. It can also be seen as the dimension of the row space or column space of the matrix.

 

Rank of a Matrix in Row Echelon Form (REF)

1. Row Echelon Form (REF): A matrix is in row echelon form when:
   • All non-zero rows are above any rows of all zeros.
   • The leading entry (pivot) of each non-zero row is to the right of the leading entry of the row above it.
   • All entries below a pivot are zero.

 

2. Finding the Rank:
   • Identify Non-Zero Rows: In REF, the rank of the matrix is equal to the number of non-zero rows. This is because each non-zero row represents a linearly independent vector in the row space of the matrix.
   • Process: Convert the matrix to REF using row operations (row swapping, scaling rows, adding/subtracting multiples of rows) and count the number of non-zero rows to determine the rank.