Okuzeka
Software Engineering vocabulary · Machine Learning and Data Processing

Corpus

Meaning in software engineering, with examples

/ˈkɔːrpəs/nounCommon term

In software engineering, corpus means: A large, organized collection of texts or speech recordings used to train language models or to study how a language is used.

In Turkish: derlem, korpus

How it is used

Engineers build, clean, filter and deduplicate a corpus, and describe it by its size in tokens, its languages and its sources. The word is more common in NLP and linguistics than in other areas of ML.

Example sentences

  1. We built a Turkish legal corpus of about two billion tokens.

  2. Half of the web corpus was duplicates, so we removed them before training.

Common collocations

  • text corpus
  • training corpus
  • build a corpus
  • parallel corpus
  • web corpus
  • datasetA collection of examples, such as images, texts or table rows, gathered and organized for training, testing or analyzing a model.
  • training dataThe examples a model learns from during training. Their quality, size and variety largely decide how well the model works on new data.
  • natural language processingThe area of AI that deals with human language, building systems that read, understand, translate, summarize or generate text and speech.
  • pretrainingThe first and largest training stage, in which a model learns general patterns from a huge amount of mostly unlabeled data before it is adapted to specific tasks.

This word in other fields

The same English word can have a different meaning in another field:

More terms: Machine Learning and Data Processing

Practise with graded reading

Reading at the right level is the fastest way to make new words stick. Pick free English books by level: