Fine-tuning

How it is used

The noun and adjective take a hyphen (fine-tuning, a fine-tuned model); the verb is fine-tune. Teams compare fine-tuning with prompting and RAG, because it costs more but can give more consistent results.

Example sentences

1. Fine-tuning on 5,000 support tickets taught the model our tone of voice.

2. Before we try fine-tuning, let's see how far we get with better prompts.

Common collocations

• fine-tuning data
• full fine-tuning
• fine-tuning job
• fine-tune a model

Related terms

• pretrainingThe first and largest training stage, in which a model learns general patterns from a huge amount of mostly unlabeled data before it is adapted to specific tasks.
• transfer learningReusing what a model learned on one task or dataset as the starting point for a different but related task, instead of training a new model from scratch.
• LoRAShort for low-rank adaptation: a fine-tuning method that keeps the original weights frozen and trains only small added matrices, which saves a lot of memory and storage.
• base modelA model that has only been pretrained, before any fine-tuning or instruction training. It continues text well but does not reliably follow instructions or chat.
• RAGShort for retrieval-augmented generation: a method in which a system first searches a document collection for relevant passages and then gives them to an LLM to write the answer.

This word in other fields

The same English word can have a different meaning in another field:

• fine-tuning · Software Engineering

More terms: Fine-Tuning and Adaptation

• RLHF
• knowledge distillation