Backpropagation

How it is used

Often shortened to backprop. It works together with gradient descent: backpropagation computes the gradients, then the optimizer updates the weights. Frameworks do it automatically, so engineers rarely write it by hand outside courses and interviews.

Example sentences

1. During backpropagation, the gradients in the early layers became almost zero.

2. Our deep learning course spends a whole week on backpropagation.

Common collocations

• backpropagation algorithm
• run backpropagation
• backpropagation through time
• compute gradients

Related terms

• gradient descentThe basic optimization method behind training: it repeatedly changes each parameter a little in the direction that reduces the loss the most.
• loss functionA formula that measures how far a model's predictions are from the correct answers. Training tries to make this value, called the loss, as small as possible.
• neural networkA model made of many connected units arranged in layers, loosely inspired by the brain. Each connection has a weight that is adjusted during training.
• weightsThe learned numbers on the connections of a neural network, which decide how strongly each input affects the next layer. Together they hold what the model has learned.

This word in other fields

The same English word can have a different meaning in another field:

• backpropagation · Software Engineering

More terms: Neural Networks and Deep Learning

• deep learning
• parameter
• transformer
• layer
• activation function
• attention