Okuzeka
Software Engineering vocabulary · Machine Learning and Data Processing

Gradient descent

Meaning in software engineering, with examples

/ˈɡreɪdiənt dɪˈsent/nounCommon term

In software engineering, gradient descent means: The basic optimization method behind training: it repeatedly changes each parameter a little in the direction that reduces the loss the most.

In Turkish: gradyan inişi, eğim inişi

How it is used

In practice people use variants: stochastic gradient descent (SGD), which works on small random batches, and optimizers such as Adam that build on it. The gradients themselves are computed by backpropagation.

Example sentences

  1. Gradient descent was barely moving because the learning rate was far too small.

  2. In the interview they asked me to explain gradient descent on a whiteboard.

Common collocations

  • stochastic gradient descent
  • run gradient descent
  • gradient descent step
  • mini-batch gradient descent
  • backpropagationThe method that calculates how much each weight in a neural network contributed to the error, by passing the error backward from the output layer toward the input.
  • learning rateA hyperparameter that controls how big a step the model takes each time it updates its parameters. Too high makes training unstable; too low makes it very slow.
  • loss functionA formula that measures how far a model's predictions are from the correct answers. Training tries to make this value, called the loss, as small as possible.
  • batch sizeThe number of training examples a model processes together before it updates its parameters once.

This word in other fields

The same English word can have a different meaning in another field:

More terms: Machine Learning and Data Processing

Practise with graded reading

Reading at the right level is the fastest way to make new words stick. Pick free English books by level: