Regularization

How it is used

Common forms are L1 and L2 regularization (weight decay), dropout and early stopping. Engineers add, increase or reduce regularization depending on whether the model overfits or underfits. The verb is regularize; the UK spelling is regularisation.

Example sentences

1. Stronger regularization closed most of the gap between training and validation accuracy.

2. We use dropout and weight decay for regularization.

Common collocations

• L2 regularization
• add regularization
• regularization strength
• weight decay

Related terms

• overfittingA problem in which a model learns its training data too closely, including noise and accidents, so it scores well on that data but poorly on new data.
• underfittingA problem in which a model is too simple or trained too little to capture the patterns in the data, so it performs poorly even on the training data.
• hyperparameterA setting chosen by engineers before training rather than learned from data, such as the learning rate, the batch size or the number of layers.
• weightsThe learned numbers on the connections of a neural network, which decide how strongly each input affects the next layer. Together they hold what the model has learned.

This word in other fields

The same English word can have a different meaning in another field:

• regularization · Software Engineering

More terms: Model Training

• train
• loss function
• epoch
• learning rate
• batch size
• gradient descent
• checkpoint
• converge