A primer on linear algebra

Fundamentals in Matrix Algebra

Matrix algebra is a branch of algebra that deals with matrices, which are rectangular arrays of numbers or symbols. Matrices are used in various areas of mathematics, science, and engineering, especially in linear algebra.

Basics of Matrices
- Definition and Elements of a Matrix
- Matrix Operations
- Matrix Transpose
- Zero Matrix and Identity Matrix
Multiplication of Matrices
Determinant of a Matrix
Inverse of a Matrix
Rank of a Matrix
Systems of Linear Equations
- Solution of Systems of Linear Equations
- Use of Inverse in Systems of Linear Equations
Eigenvalues and Eigenvectors of a Matrix
Orthogonal and Unitary Matrices
Matrix Decomposition
Matrix Diagonalization
Applications of Matrices
- Applications in Computer Graphics
- Applications in Network Theory
Vector Spaces

Matrix Notation

A matrix is typically denoted by an uppercase letter, such as $A$ , and its elements are enclosed in square brackets. For example,

$A = [\begin{matrix} a_{11} & a_{12} & \dots & a_{1 n} \\ a_{21} & a_{22} & \dots & a_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ a_{m 1} & a_{m 2} & \dots & a_{m n} \end{matrix}]$

Where $A$ is an $m \times n$ matrix with $a_{i j}$ as its element in the $i$ -th row and $j$ -th column.

Matrix Operations

Addition and Subtraction

Matrices of the same dimensions can be added or subtracted element-wise. For example,

$A + B = [\begin{matrix} a_{11} + b_{11} & a_{12} + b_{12} & \dots & a_{1 n} + b_{1 n} \\ a_{21} + b_{21} & a_{22} + b_{22} & \dots & a_{2 n} + b_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ a_{m 1} + b_{m 1} & a_{m 2} + b_{m 2} & \dots & a_{m n} + b_{m n} \end{matrix}]$

$A - B = [\begin{matrix} a_{11} - b_{11} & a_{12} - b_{12} & \dots & a_{1 n} - b_{1 n} \\ a_{21} - b_{21} & a_{22} - b_{22} & \dots & a_{2 n} - b_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ a_{m 1} - b_{m 1} & a_{m 2} - b_{m 2} & \dots & a_{m n} - b_{m n} \end{matrix}]$

Scalar Multiplication

A matrix can be multiplied by a scalar, which multiplies each element of the matrix by the scalar value. For example,

$α A = [\begin{matrix} α a_{11} & α a_{12} & \dots & α a_{1 n} \\ α a_{21} & α a_{22} & \dots & α a_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ α a_{m 1} & α a_{m 2} & \dots & α a_{m n} \end{matrix}]$

Where $α$ is a scalar.

Matrix Multiplication

Matrix multiplication is more complex than addition or scalar multiplication. To multiply two matrices together, the number of columns in the first matrix must equal the number of rows in the second matrix.

Given two matrices $A$ and $B$ with dimensions $m \times n$ and $n \times p$ respectively, the resulting matrix $C$ will have dimensions $m \times p$ .

The element at position $(i, j)$ in $C$ is calculated by multiplying the $i$ -th row of $A$ with the $j$ -th column of $B$ and summing the products. Mathematically, the matrix multiplication can be represented as:

$C = A \cdot B$

$C_{i j} = a_{i 1} b_{1 j} + a_{i 2} b_{2 j} + \dots + a_{i n} b_{n j} = \sum_{k = 1}^{n} a_{i k} b_{k j}$

Properties of Matrix Algebra

Matrix algebra obeys several properties, similar to real number algebra. These properties include:

Rule	Example
Associativity	$(A B) C = A (B C)$
Distributivity	$A (B + C) = A B + A C$
Scalar Multiplication	$α (A + B) = α A + α B$
Identity Matrix	$A I = I A = A$
Zero Matrix	$A 0 = 0 A = 0$

These properties play an important role in solving equations, performing transformations, and solving systems of linear equations using matrix algebra.

Matrix Transposition

Matrix transposition is a special operation in linear algebra that flips the rows and columns of a matrix over its main diagonal, which runs down from the top-left to bottom-right. A transposition is represented with a superscript “top” symbol or “’”.

If we have a matrix $A$ , its transpose is represented as $A^{⊤}$ . Formally, if a matrix $A$ is of size $m \times n$ (m rows, n columns), then the transpose of matrix $A$ , denoted by $A^{⊤}$ or just $A^{'}$ , is of size $n \times m$ (n rows, m columns). For each $i$ and $j$ , the elements of matrix $A^{⊤}$ are given by:

$(A^{⊤})_{i j} = A_{j i}$

Simply put, what was in the j-th row and i-th column of the original matrix, will be in the i-th row and j-th column of the transposed matrix. Let’s see how this works: Consider the matrix $A = [\begin{matrix} a & b \\ c & d \\ e & f \end{matrix}]$ . The transpose matrix $A^{⊤}$ is given by:

$A^{⊤} = [\begin{matrix} a & c & e \\ b & d & f \end{matrix}]$

Transpose Properties

Here are some important properties of matrix transposition:

Property	Equation
Transpose of a Transpose	$(A^{⊤})^{⊤} = A$
Transpose of a Sum	$(A + B)^{⊤} = A^{⊤} + B^{⊤}$
Transpose of a Difference	$(A - B)^{⊤} = A^{⊤} - B^{⊤}$
Transpose of a Scalar Multiplication	$(c A)^{⊤} = c (A^{⊤})$
Transpose of a Product	$(A B)^{⊤} = B^{⊤} A^{⊤}$

The first four properties are straightforward. The last property, transpose of a product, is a bit more complex because when taking the transpose of a product of matrices, the order of the matrices is reversed in the product. This property is not intuitive and needs to proven with matrix algebra.

Also, remember that matrix multiplication is not commutative ( $A B \neq B A$ generally); hence the reversal in the order of multiplication is very important.

--- title: A primer on linear algebra --- ## Fundamentals in Matrix Algebra Matrix algebra is a branch of algebra that deals with matrices, which are rectangular arrays of numbers or symbols. Matrices are used in various areas of mathematics, science, and engineering, especially in linear algebra. 1. Basics of Matrices - Definition and Elements of a Matrix - Matrix Operations - Matrix Transpose - Zero Matrix and Identity Matrix 2. Multiplication of Matrices 3. Determinant of a Matrix 4. Inverse of a Matrix 5. Rank of a Matrix 6. Systems of Linear Equations - Solution of Systems of Linear Equations - Use of Inverse in Systems of Linear Equations 7. Eigenvalues and Eigenvectors of a Matrix 8. Orthogonal and Unitary Matrices 9. Matrix Decomposition 10. Matrix Diagonalization 11. Applications of Matrices - Applications in Computer Graphics - Applications in Network Theory 12. Vector Spaces ### Matrix Notation A matrix is typically denoted by an uppercase letter, such as $\mathbf A$, and its elements are enclosed in square brackets. For example, $$ \mathbf{A} = \begin{bmatrix} a_{11} & a_{12} & \ldots & a_{1n} \\ a_{21} & a_{22} & \ldots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \ldots & a_{mn} \\ \end{bmatrix} $$ Where $\mathbf A$ is an $m \times n$ matrix with $a_{ij}$ as its element in the $i$-th row and $j$-th column. ### Matrix Operations #### Addition and Subtraction Matrices of the same dimensions can be added or subtracted element-wise. For example, $$ \mathbf{A} + \mathbf{B} = \begin{bmatrix} a_{11}+b_{11} & a_{12}+b_{12} & \ldots & a_{1n}+b_{1n} \\ a_{21}+b_{21} & a_{22}+b_{22} & \ldots & a_{2n}+b_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1}+b_{m1} & a_{m2}+b_{m2} & \ldots & a_{mn}+b_{mn} \\ \end{bmatrix} $$ $$ \mathbf{A} - \mathbf{B} = \begin{bmatrix} a_{11}-b_{11} & a_{12}-b_{12} & \ldots & a_{1n}-b_{1n} \\ a_{21}-b_{21} & a_{22}-b_{22} & \ldots & a_{2n}-b_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1}-b_{m1} & a_{m2}-b_{m2} & \ldots & a_{mn}-b_{mn} \\ \end{bmatrix} $$ #### Scalar Multiplication A matrix can be multiplied by a scalar, which multiplies each element of the matrix by the scalar value. For example, $$ \alpha \mathbf{A} = \begin{bmatrix} \alpha a_{11} & \alpha a_{12} & \ldots & \alpha a_{1n} \\ \alpha a_{21} & \alpha a_{22} & \ldots & \alpha a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ \alpha a_{m1} & \alpha a_{m2} & \ldots & \alpha a_{mn} \\ \end{bmatrix} $$ Where $\alpha$ is a scalar. #### Matrix Multiplication Matrix multiplication is more complex than addition or scalar multiplication. To multiply two matrices together, the number of columns in the first matrix must equal the number of rows in the second matrix. Given two matrices $\mathbf A$ and $\mathbf B$ with dimensions $m \times n$ and $n \times p$ respectively, the resulting matrix $\mathbf C$ will have dimensions $m \times p$. The element at position $(i,j)$ in $\mathbf C$ is calculated by multiplying the $i$-th row of $\mathbf A$ with the $j$-th column of $\mathbf B$ and summing the products. Mathematically, the matrix multiplication can be represented as: $$ \mathbf{C} = \mathbf{A} \cdot \mathbf{B} $$ $$ C_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \ldots + a_{in}b_{nj} = \sum_{k=1}^{n} a_{ik}b_{kj} $$ ### Properties of Matrix Algebra Matrix algebra obeys several properties, similar to real number algebra. These properties include: | Rule | Example | |-|-| | Associativity | $(\mathbf{A} \mathbf{B}) \mathbf{C} = \mathbf{A} (\mathbf{B} \mathbf{C})$ | | Distributivity | $\mathbf{A} (\mathbf{B} + \mathbf{C}) = \mathbf{A} \mathbf{B} + \mathbf{A} \mathbf{C}$ | | Scalar Multiplication | $\alpha (\mathbf{A} + \mathbf{B}) = \alpha \mathbf{A} + \alpha \mathbf{B}$ | | Identity Matrix | $\mathbf{A} \mathbf{I} = \mathbf{I} \mathbf{A} = \mathbf{A}$ | | Zero Matrix | $\mathbf{A} \mathbf{0} = \mathbf{0} \mathbf{A} = \mathbf{0}$ | These properties play an important role in solving equations, performing transformations, and solving systems of linear equations using matrix algebra. ### Matrix Transposition Matrix transposition is a special operation in linear algebra that flips the rows and columns of a matrix over its main diagonal, which runs down from the top-left to bottom-right. A transposition is represented with a superscript "top" symbol or "'". If we have a matrix $\mathbf{A}$, its transpose is represented as $\mathbf{A}^\top$. Formally, if a matrix $\mathbf{A}$ is of size $m \times n$ (m rows, n columns), then the transpose of matrix $\mathbf{A}$, denoted by $\mathbf{A}^\top$ or just $\mathbf{A}'$, is of size $n \times m$ (n rows, m columns). For each $i$ and $j$, the elements of matrix $\mathbf{A}^\top$ are given by: $$ (\mathbf{A}^\top)_{ij} = \mathbf{A}_{ji} $$ Simply put, what was in the j-th row and i-th column of the original matrix, will be in the i-th row and j-th column of the transposed matrix. Let's see how this works: Consider the matrix $\mathbf{A} = \begin{bmatrix} a & b \\ c & d \\ e & f \end{bmatrix}$. The transpose matrix $\mathbf{A}^\top$ is given by: $$ \mathbf{A}^\top = \begin{bmatrix} a & c & e \\ b & d & f \end{bmatrix} $$ #### Transpose Properties Here are some important properties of matrix transposition: | Property | Equation | | ------------------------------------ | -------------------------------------------------------------------- | | Transpose of a Transpose | $(\mathbf{A}^\top)^\top = \mathbf{A}$ | | Transpose of a Sum | $(\mathbf{A} + \mathbf{B})^\top = \mathbf{A}^\top + \mathbf{B}^\top$ | | Transpose of a Difference | $(\mathbf{A} - \mathbf{B})^\top = \mathbf{A}^\top - \mathbf{B}^\top$ | | Transpose of a Scalar Multiplication | $(c \mathbf{A})^\top = c (\mathbf{A}^\top)$ | | Transpose of a Product | $(\mathbf{A} \mathbf{B})^\top = \mathbf{B}^\top \mathbf{A}^\top$ | The first four properties are straightforward. The last property, transpose of a product, is a bit more complex because when taking the transpose of a product of matrices, the order of the matrices is reversed in the product. This property is not intuitive and needs to proven with matrix algebra. Also, remember that matrix multiplication is not commutative ($\mathbf{A} \mathbf{B} \neq \mathbf{B} \mathbf{A}$ generally); hence the reversal in the order of multiplication is very important.