Linear Algebra

Matrix

A rectangular array of numbers written between square brackets. These could be features from a learning problem or it could be data from somewhere.

Vector

A vector is a matrix that has only 1 column so we have an N x 1 matrix where N is the number of rows, and 1 is number of columns. So matrix with just one column is what we call a vector.

Note:
• By convention, usually when writing matrices and vectors, most people use upper case to refer to matrices, like A, B, C or X.
• And lowercase, like a, b, x, y to refer to either numbers, or just raw numbers or scalars or to vectors
Matrix Representations: Code & Output
% The ; denotes we are going back to a new row.
A = [1, 2, 3; 4, 5, 6; 7, 8, 9; 10, 11, 12]

% Initialize a vector
v = [1;2;3]

% Get the dimension of the matrix A where m = rows and n = columns
[m,n] = size(A)

% You could also store it this way
dim_A = size(A)

% Get the dimension of the vector v
dim_v = size(v)

% Now let's index into the 2nd row 3rd column of matrix A
A_23 = A(2,3)

Matrix Operations

Note:

We can add/subtract matrices of only same dimension. The result of addition will be a matrix of the same dimension.

Matrix Operations: Code and Output
% Initialize matrix A and B
A = [1, 2, 4; 5, 3, 2]
B = [1, 3, 4; 1, 1, 1]

% Initialize constant s
s = 2

% See how element-wise addition works

% See how element-wise subtraction works
sub_AB = A - B

% See how scalar multiplication works
mult_As = A * s

% Divide A by s
div_As = A / s

% What happens if we have a Matrix + scalar?

Matrix-Vector Mutliplication

Matrix-Vector Mutliplication: Code and Output
% Initialize matrix A
A = [1, 2, 3; 4, 5, 6;7, 8, 9]

% Initialize vector v
v = [1; 1; 1]

% Multiply A * v
Av = A * v

Matrix-Matrix Multiplication

Notes:
• Number of columns in first matrix should match the number of rows in the second matrix.
• When an m×n matrix is multiplied with n×p matrix the resultant matrix is of dimension m×p.

Matrix-Matrix Mutliplication: Code and Output
% Initialize a 3 by 2 matrix
A = [1, 2; 3, 4;5, 6]

% Initialize a 2 by 1 matrix
B = [1; 2]

% We expect a resulting matrix of (3 by 2)*(2 by 1) = (3 by 1)
mult_AB = A*B

Matrix Multiplication Properties

1. Matrices are not commutative: A∗B≠B∗A

2. Matrices are associative: (A∗B)∗C=A∗(B∗C)

3. Multiplication of any matrix with Identity matrix results in original matrix. It’s just like multiplying numbers by 1.

Matrix Mutliplication Properties: Code and Output
% Initialize random matrices A and B
A = [1,2;4,5]
B = [1,1;0,2]

% Initialize a 2 by 2 identity matrix
I = eye(2)

% The above notation is the same as I = [1,0;0,1]

% What happens when we multiply I*A ?
IA = I*A

AI = A*I

% Compute A*B
AB = A*B

% Is it equal to B*A?
BA = B*A

% Note that IA = AI but AB != BA

Inverse & Transpose of Matrix

Matrix Inverse:

Notes:
• Matrices that don’t have inverse are “singular” or “degenerate”.
• Sometimes we can compute inverses by hand but almost no one does that these days, we use some softwares like octave/matlab.

Matrix Transpose:

Inverse & Transpose of Matrix: Code and Output
% Initialize matrix A
A = [1,2,0;0,5,6;7,0,9]

% Transpose A
A_trans = A'

% Take the inverse of A
A_inv = inv(A)

% What is A^(-1)*A?
A_invA = inv(A)*A