DSAverse

Initializing Sorting Algorithms...

Sorting

Trees

Graphs

Preparing interactive visualizations...

Back to Machine Learning

Overfitting and Underfitting

Slide a polynomial from degree 0 to 11 on twelve noisy points. Training error only ever falls; validation error traces a U. An L2 penalty pulls the wild coefficients back in.

Builds onLinear Regression Loss Functions

Every fit solved in your browser · seeded noisy samples of a known curve

Act 1 of 5 · The datastep 1 of 30

Speed

Twelve noisy measurements of a hidden curve. We will fit polynomials to them: degree d means d + 1 coefficients, ŷ = c₀ + c₁x + c₂x² + … up to the xᵈ term. The more coefficients, the more the curve can bend.

\hat y = c_0 + c_1 x + c_2 x^2 + \dots + c_d x^d, \qquad X \in \mathbb{R}^{12 \times (d+1)}

Reading the two errors

Both high — underfitting: the model is too simple (high bias).
Training low, validation high — overfitting: it learned the noise (high variance).
Both low, close together — the bottom of the U.

X[n, d+1] =[12, 12]c[d+1] =[12]

A common misconception

“A model that fits its training data perfectly has learned the pattern.”

This page fitted degree 11 to 200 seeded datasets of twelve points. Every fit was perfect on its training points (worst training MSE 1.7 × 10⁻²⁴), and on validation it did worse than plain degree 3 in 199 of them — typically 14 times worse.

Check yourself

Question 1 of 3

As the polynomial degree rises from 0 to 11 on twelve training points, what happens to training error?

Overfitting and Underfitting

Slide a polynomial from degree 0 to 11 on twelve noisy points. Training error only ever falls; validation error traces a U. An L2 penalty pulls the wild coefficients back in.

Builds onLinear Regression Loss Functions

Every fit solved in your browser · seeded noisy samples of a known curve

Act 1 of 5 · The datastep 1 of 30

Speed

\hat y = c_0 + c_1 x + c_2 x^2 + \dots + c_d x^d, \qquad X \in \mathbb{R}^{12 \times (d+1)}

Reading the two errors

Both high — underfitting: the model is too simple (high bias).
Training low, validation high — overfitting: it learned the noise (high variance).
Both low, close together — the bottom of the U.

X[n, d+1] =[12, 12]c[d+1] =[12]

A common misconception

“A model that fits its training data perfectly has learned the pattern.”

Check yourself

Question 1 of 3

As the polynomial degree rises from 0 to 11 on twelve training points, what happens to training error?

DSAverse

Overfitting and Underfitting

See also

Overfitting and Underfitting

See also