Okuzeka
Artificial Intelligence & Machine Learning vocabulary · Bias, Safety and Responsible AI

Red teaming

Meaning in artificial intelligence & machine learning, with examples

nounAdvanced term

In artificial intelligence & machine learning, red teaming means: Testing a model or AI system by deliberately trying to make it fail or behave harmfully, in order to find weaknesses before real users or attackers do.

In Turkish: kırmızı takım testi, saldırgan gözüyle test

How it is used

Done by internal teams, outside experts or automated tools, before and after launch. The people involved are red teamers, and their findings feed into guardrails and further training. The term comes from security exercises in which one team plays the attacker.

Example sentences

  1. Red teaming showed that the assistant would reveal its system prompt if asked in French.

  2. We planned two weeks of red teaming before the public launch.

Common collocations

  • red teaming exercise
  • red-team a model
  • red teamers
  • automated red teaming
  • jailbreakA prompt or technique designed to trick a model into ignoring its safety rules and producing content it is meant to refuse.
  • guardrailA rule or check placed around a model to stop harmful, off-topic or unsafe inputs and outputs, for example a filter that blocks personal data in answers.
  • alignmentMaking an AI system's behavior match human intentions and values, so that it is helpful and honest and avoids causing harm. Also the research field that studies this.
  • biasSystematic unfairness in a model's outputs, for example working worse for some groups of people, usually caused by skewed data or design choices.

This word in other fields

The same English word can have a different meaning in another field:

More terms: Bias, Safety and Responsible AI

Practise with graded reading

Reading at the right level is the fastest way to make new words stick. Pick free English books by level: