Transformer

How it is used

Used as a noun and before other nouns: a transformer model, transformer layers, a transformer-based system. Engineers talk about encoder-only, decoder-only and encoder-decoder transformers. They are used for images and audio too, not only text.

Example sentences

1. We replaced the old recurrent model with a small transformer and cut errors by a third.

2. Most current LLMs are decoder-only transformers.

Common collocations

• transformer model
• transformer architecture
• decoder-only transformer
• transformer-based

Related terms

• attentionA mechanism that lets a model decide which other parts of its input are most relevant to each part, and give those parts more weight when it builds its output.
• LLMShort for large language model: a very large neural network trained on huge amounts of text to predict the next token, which lets it answer questions, write, summarize and code.
• layerOne level of a neural network: a group of units that takes the output of the previous level, transforms it and passes the result on to the next one.
• neural networkA model made of many connected units arranged in layers, loosely inspired by the brain. Each connection has a weight that is adjusted during training.

This word in other fields

The same English word can have a different meaning in another field:

• transformer · Artificial Intelligence & Machine Learning
• transformer · Electrical & Electronics Engineering
• transformer · Energy & Renewables

More terms: Machine Learning and Data Processing

• artificial intelligence
• machine learning
• model
• dataset
• training data
• label
• train
• supervised learning
• deep learning
• loss function
• overfitting
• test set