You need to login before you can make any post submission.

Add New Chapter
## Tutorial

# Regularization

## Solving the Problem of Overfitting

### The Problem of Overfitting

### Cost Function

##### Intuition:

##### Regularizing all θ parameters

###### Note: For regularization to work properly λ should be choosen appropriately.

### Regularized Linear Regression

#### Gradient Descent

#### Normal Equation

##### Note:

### Regularized Logistic Regression

#### Gradient Descent

#### Advanced Optimization

Introduction to Machine Learning

Linear Regression with One Variable

Linear Algebra Review

Linear Regression with Multiple Variables

Octave / Matlab Tutorial

Logistic Regression

Regularization

Neural Networks

Applying Machine Learning

Machine Learning System Design

Support Vector Machines

Unsupervised Learning

Dimensionality Reduction

Anomaly Detection

Recommender Systems

Large Scale Machine Learning

Example Application - Photo OCR

Linear Regression with One Variable

Linear Algebra Review

Linear Regression with Multiple Variables

Octave / Matlab Tutorial

Logistic Regression

Regularization

Neural Networks

Applying Machine Learning

Machine Learning System Design

Support Vector Machines

Unsupervised Learning

Dimensionality Reduction

Anomaly Detection

Recommender Systems

Large Scale Machine Learning

Example Application - Photo OCR

- Machine learning models need to generalize well to new examples that the model has not seen in practice.
- Regularization helps prevent models from overfitting the training data.

When we apply machine learining algorithms to certain machine learning applications, they can run into a problem called overfitting that can cause them to perform very poorly.

Overfitting Examples:

Addressing Overfitting:

If we have overfitting from hypothesis function, we can reduce the weight that some of the terms in function carry by increasing their cost.

Say we wanted to make the following function more quadratic:

θ_{0 }+ θ_{1}x + θ_{2}x^{2}+ θ_{3}x^{3}+ θ_{4}x^{4}

We’ll want to eliminate the influence of **θ _{3}x^{3} + θ_{4}x^{4}** but without actually getting rid of these features or changing the form of our hypothesis, we can instead modify our

The **λ** is the **regularization parameter**. It determines how much the costs of our theta parameters are inflated.

Using the cost function with the extra summation, we can smooth the output of our hypothesis function to reduce overfitting.

If lambda is chosen to be too large, it may smooth out the function too much and cause underfitting.

We can apply regularization to both linear regression and logistic regression.

We will modify our gradient descent function to separate out **θ _{0}** from the rest of the parameters because we do not want to penalize

Concept:

Regularization can also be appraoched using the alternate method of the non-iterative normal equation.

To add in regularization, the equation is the same as our original, except that we add another term inside the parentheses:

- If
**m < n**, then**X**. However, when we add the term^{T}X is non-invertible**λ⋅L**, then**X**becomes invertible.^{T}X + λ⋅L

We can regularize logistic regression in a similar way that we regularize linear regression to avoid overfitting.

Concept: