Catastrophic forgetting

How it is used

Used in continual learning and fine-tuning. Better performance on the new task alone can hide losses on older tasks.

Example sentences

1. Fine-tuning on short commands caused catastrophic forgetting of the model's earlier translation skill.

   Learning the new task damaged an ability that had already been acquired.

2. We kept an older task benchmark to detect catastrophic forgetting after each training stage.

   Repeated evaluation checks whether earlier capabilities survive the additional training.

Common collocations

• mitigate catastrophic forgetting
• catastrophic forgetting in neural networks

Related terms

• fine-tuningExtra training of an already pretrained model on a smaller, task-specific dataset, so that it does better on that task or follows a particular style.
• benchmarkA standard set of tasks and data used to measure models in the same way, so that results from different teams can be compared fairly.

More terms: Machine Learning and Data Processing

• artificial intelligence
• machine learning
• model
• dataset
• training data
• label
• train
• supervised learning
• neural network
• deep learning
• loss function
• overfitting