Machine Learning Fundamentals · Model evaluation

Overfitting and underfitting

A model that memorizes training data instead of learning the pattern (overfit), or one too weak to learn it at all (underfit).

Also known as: generalization gap.

Start a lesson on Overfitting and underfitting

Learn first

• Bias-variance tradeoff: A model too simple has high bias (underfits); one too flexible has high variance (overfits), and you can't minimize both at once.

Leads to

• Regularization (L1/L2): Penalizing large model weights during training so it can't overfit by memorizing noise.

Related topics

• Regularization (L1/L2): Penalizing large model weights during training so it can't overfit by memorizing noise.
• Cross-validation: Splitting the data multiple different ways and averaging performance, so one lucky/unlucky split doesn't mislead you.