Machine Learning Fundamentals · Foundations

Gradient descent

Repeatedly nudging a model's parameters downhill on the cost function to reduce error.

Also known as: SGD, learning rate, optimization.

Start a lesson on Gradient descent

Learn first

• Cost functions and loss: The single number a model tries to minimize during training, chosen to reflect how wrong its predictions are.

Leads to

• Logistic regression: A linear model adapted for classification by squashing its output into a probability.
• Neural network basics: Stacking layers of logistic-regression-like units, connected by non-linear activation functions, to learn more complex patterns.

Related topics

• Neural network basics: Stacking layers of logistic-regression-like units, connected by non-linear activation functions, to learn more complex patterns.