Alignment

How it is used

Used both for practical work (RLHF, safety training, evaluations) and for long-term research on more capable systems. People say a model is well aligned or misaligned, and talk about alignment teams and alignment research.

Example sentences

1. Alignment work made the model more willing to admit when it doesn't know something.

2. She left product engineering to join the alignment research team.

Common collocations

• alignment research
• alignment team
• well aligned
• misaligned model
• safety and alignment

Related terms

• RLHFShort for reinforcement learning from human feedback: a training stage in which people rank model answers, a reward model learns their preferences, and the LLM is then tuned to score higher.
• guardrailA rule or check placed around a model to stop harmful, off-topic or unsafe inputs and outputs, for example a filter that blocks personal data in answers.
• red teamingTesting a model or AI system by deliberately trying to make it fail or behave harmfully, in order to find weaknesses before real users or attackers do.
• biasSystematic unfairness in a model's outputs, for example working worse for some groups of people, usually caused by skewed data or design choices.

This word in other fields

The same English word can have a different meaning in another field:

• alignment · Artificial Intelligence & Machine Learning
• alignment · UX/UI Design
• alignment · Product Management
• alignment · Genetics & Biotechnology
• alignment · Management & Leadership
• alignment · Graphic Design

More terms: Machine Learning and Data Processing

• artificial intelligence
• machine learning
• model
• dataset
• training data
• label
• train
• supervised learning
• neural network
• deep learning
• loss function
• overfitting