Base model

How it is used

Contrasted with instruct or chat models, which have been tuned to follow instructions. Teams pick a base model as the starting point for their own fine-tuning. In general talk, 'base model' can also just mean the model you start from.

Example sentences

1. We fine-tuned the base model, not the chat version, because we needed a very specific output format.

2. The base model just continued my question instead of answering it.

Common collocations

• choose a base model
• base model checkpoint
• base versus instruct model
• fine-tune the base model

Related terms

• pretrainingThe first and largest training stage, in which a model learns general patterns from a huge amount of mostly unlabeled data before it is adapted to specific tasks.
• fine-tuningExtra training of an already pretrained model on a smaller, task-specific dataset, so that it does better on that task or follows a particular style.
• checkpointA saved copy of a model's weights, and often its optimizer state, at a certain point in training, so that training can be resumed or that version can be used later.
• LLMShort for large language model: a very large neural network trained on huge amounts of text to predict the next token, which lets it answer questions, write, summarize and code.

This word in other fields

The same English word can have a different meaning in another field:

• base model · Software Engineering

More terms: Fine-Tuning and Adaptation

• transfer learning
• LoRA
• RLHF
• knowledge distillation