DSAverse

Initializing Sorting Algorithms...

Sorting

Trees

Graphs

Preparing interactive visualizations...

Back to Machine Learning

Loss Functions

The loss decides what “good” means. Same points, three losses, three different best lines — and a separate loss for classifiers that never lets confident mistakes off lightly.

Builds onLinear Regression What Is Machine Learning?

Every fit solved exactly in your browser · seeded example points

Act 1 of 7 · Residualsstep 1 of 21

Squared· grab a point to take over

Speed

Ten points and a line. Each point's residual r = ŷ − y is how far the line misses it — the dashed red gaps, here from −1.44 to 2.23. A loss function turns those ten residuals into one number, and training means finding the line with the smallest number. Which loss you pick decides which line counts as best.

\text{MSE} = \tfrac{1}{10}\textstyle\sum_{i=1}^{10} r_i^2 = 0.8859, \quad r_i = (0.437)\,x_i + 0.287 - y_i

Four losses

Squared (MSE) — mean r². Smooth and easy to solve; one big miss dominates.
Absolute (MAE) — mean |r|. Robust to outliers; its best constant is the median.
Huber — squared inside δ, absolute outside. The usual compromise.
Cross-entropy — −ln p of the true class. The loss for classifiers.

A common misconception

“The loss is just a scoreboard — any sensible one finds the same best line.”

The loss is the definition of best. This page built 300 seeded datasets like the tour's and moved one end point by 3.4. On average the squared-loss slope moved by 0.267, the Huber slope by 0.155 and the absolute-loss slope by 0.118: same points, three different answers.

Check yourself

Question 1 of 3

One data point is a wild outlier. Which loss’s best-fit line moves the most towards it?

Loss Functions

The loss decides what “good” means. Same points, three losses, three different best lines — and a separate loss for classifiers that never lets confident mistakes off lightly.

Builds onLinear Regression What Is Machine Learning?

Every fit solved exactly in your browser · seeded example points

Act 1 of 7 · Residualsstep 1 of 21

Squared· grab a point to take over

Speed

\text{MSE} = \tfrac{1}{10}\textstyle\sum_{i=1}^{10} r_i^2 = 0.8859, \quad r_i = (0.437)\,x_i + 0.287 - y_i

Four losses

Squared (MSE) — mean r². Smooth and easy to solve; one big miss dominates.
Absolute (MAE) — mean |r|. Robust to outliers; its best constant is the median.
Huber — squared inside δ, absolute outside. The usual compromise.
Cross-entropy — −ln p of the true class. The loss for classifiers.

A common misconception

“The loss is just a scoreboard — any sensible one finds the same best line.”

Check yourself

Question 1 of 3

One data point is a wild outlier. Which loss’s best-fit line moves the most towards it?

DSAverse

Loss Functions

See also

Loss Functions

See also