Machine Learning

Add New Chapter

Tutorial

Linear Algebra

Matrix

A rectangular array of numbers written between square brackets. These could be features from a learning problem or it could be data from somewhere.

matrix_representation__85.

Vector

A vector is a matrix that has only 1 column so we have an N x 1 matrix where N is the number of rows, and 1 is number of columns. So matrix with just one column is what we call a vector.

vector_representation__40.

Note:
  • By convention, usually when writing matrices and vectors, most people use upper case to refer to matrices, like A, B, C or X.
  • And lowercase, like a, b, x, y to refer to either numbers, or just raw numbers or scalars or to vectors
Matrix Representations: Code & Output
% The ; denotes we are going back to a new row.
A = [1, 2, 3; 4, 5, 6; 7, 8, 9; 10, 11, 12]

% Initialize a vector 
v = [1;2;3] 

% Get the dimension of the matrix A where m = rows and n = columns
[m,n] = size(A)

% You could also store it this way
dim_A = size(A)

% Get the dimension of the vector v 
dim_v = size(v)

% Now let's index into the 2nd row 3rd column of matrix A
A_23 = A(2,3)

Matrix Operations

matrix_operations__80.


Note:

We can add/subtract matrices of only same dimension. The result of addition will be a matrix of the same dimension.

Matrix Operations: Code and Output
% Initialize matrix A and B 
A = [1, 2, 4; 5, 3, 2]
B = [1, 3, 4; 1, 1, 1]

% Initialize constant s 
s = 2

% See how element-wise addition works
add_AB = A + B 

% See how element-wise subtraction works
sub_AB = A - B

% See how scalar multiplication works
mult_As = A * s

% Divide A by s
div_As = A / s

% What happens if we have a Matrix + scalar?
add_As = A + s

Matrix-Vector Mutliplication

matrix_vector_multiplication__80.


matrix_vector_multiply_examples__75.

Matrix-Vector Mutliplication: Code and Output
% Initialize matrix A 
A = [1, 2, 3; 4, 5, 6;7, 8, 9] 

% Initialize vector v 
v = [1; 1; 1] 

% Multiply A * v
Av = A * v

Matrix-Matrix Multiplication

matrix_matrix_multiplication__85.

Notes:
  • Number of columns in first matrix should match the number of rows in the second matrix.
  • When an m×n matrix is multiplied with n×p matrix the resultant matrix is of dimension m×p.


matrix_matrix_multiplication_examples__65.

Matrix-Matrix Mutliplication: Code and Output
% Initialize a 3 by 2 matrix 
A = [1, 2; 3, 4;5, 6]

% Initialize a 2 by 1 matrix 
B = [1; 2] 

% We expect a resulting matrix of (3 by 2)*(2 by 1) = (3 by 1) 
mult_AB = A*B

Matrix Multiplication Properties

1. Matrices are not commutative: A∗B≠B∗A

matrix_commutative_property__35.


2. Matrices are associative: (A∗B)∗C=A∗(B∗C)

matrix_associative_property__35.


3. Multiplication of any matrix with Identity matrix results in original matrix. It’s just like multiplying numbers by 1.

identity_matrix_property__45.


Matrix Mutliplication Properties: Code and Output
% Initialize random matrices A and B 
A = [1,2;4,5]
B = [1,1;0,2]

% Initialize a 2 by 2 identity matrix
I = eye(2)

% The above notation is the same as I = [1,0;0,1]

% What happens when we multiply I*A ? 
IA = I*A 

% How about A*I ? 
AI = A*I 

% Compute A*B 
AB = A*B 

% Is it equal to B*A? 
BA = B*A 

% Note that IA = AI but AB != BA

Inverse & Transpose of Matrix

matrix_inverse_transpose__85.

Matrix Inverse:

inverse_of_matrix__40.

Notes:
  • Matrices that don’t have inverse are “singular” or “degenerate”.
  • Sometimes we can compute inverses by hand but almost no one does that these days, we use some softwares like octave/matlab.

Matrix Transpose:

transpose_of_matrix__35.


Inverse & Transpose of Matrix: Code and Output
% Initialize matrix A 
A = [1,2,0;0,5,6;7,0,9]

% Transpose A 
A_trans = A' 

% Take the inverse of A 
A_inv = inv(A)

% What is A^(-1)*A? 
A_invA = inv(A)*A