Machine Learning Fundamentals · Model evaluation

Hyperparameter tuning

Systematically searching for the settings (learning rate, regularization strength, tree depth) that aren't learned from data but chosen before training.

Also known as: grid search, random search.

Start a lesson on Hyperparameter tuning

Learn first

• Cross-validation: Splitting the data multiple different ways and averaging performance, so one lucky/unlucky split doesn't mislead you.
• Regularization (L1/L2): Penalizing large model weights during training so it can't overfit by memorizing noise.

Related topics

• Backpropagation: The algorithm that computes how much each weight in a neural network contributed to the error, so gradient descent knows which way to adjust it.