Okuzeka
Artificial Intelligence & Machine Learning vocabulary · Model Training

Learning rate

Meaning in artificial intelligence & machine learning, with examples

nounCore term

In artificial intelligence & machine learning, learning rate means: A hyperparameter that controls how big a step the model takes each time it updates its parameters. Too high makes training unstable; too low makes it very slow.

In Turkish: öğrenme oranı, öğrenme hızı

How it is used

Usually the first hyperparameter people tune. It is often changed during training by a learning rate schedule, with a short warm-up at the start and a gradual decay later. Written as LR in notes and code, for example 'LR of 3e-4'.

Example sentences

  1. The loss exploded after a few hundred steps, so we lowered the learning rate.

  2. We increase the learning rate slowly during the first 1,000 steps as a warm-up.

Common collocations

  • lower the learning rate
  • learning rate schedule
  • learning rate warm-up
  • initial learning rate
  • hyperparameterA setting chosen by engineers before training rather than learned from data, such as the learning rate, the batch size or the number of layers.
  • gradient descentThe basic optimization method behind training: it repeatedly changes each parameter a little in the direction that reduces the loss the most.
  • epochOne complete pass of the training process through the whole training dataset. Models are usually trained for several epochs.
  • batch sizeThe number of training examples a model processes together before it updates its parameters once.

This word in other fields

The same English word can have a different meaning in another field:

More terms: Model Training

Practise with graded reading

Reading at the right level is the fastest way to make new words stick. Pick free English books by level: