Benchmark

How it is used

Models are evaluated on, scored on and compared across benchmarks, and a new best result tops a benchmark or a leaderboard. It is also a verb: to benchmark a model. Engineers often warn that high benchmark scores don't always mean good real-world performance.

Example sentences

1. It scores well on public benchmarks but struggles with our customers' questions.

2. We built an internal benchmark from 500 real support conversations.

Common collocations

• public benchmark
• internal benchmark
• benchmark score
• top a benchmark
• benchmark results

Related terms

• state-of-the-artDescribes the best result or method known at a given time, usually measured on a well-known benchmark.
• test setA part of the data kept aside and used only at the end, to measure how well a finished model works on examples it has never seen.
• accuracyThe share of a model's predictions that are correct, out of all its predictions. A model with 90% accuracy is right on nine of every ten examples.
• data leakageA mistake in which information that won't be available at prediction time, such as test examples or future data, gets into training and makes results look better than they are.

This word in other fields

The same English word can have a different meaning in another field:

• benchmark · Software Engineering
• benchmark · Civil Engineering & Construction
• benchmark · Healthcare Management
• benchmark · Management & Leadership
• benchmark · Investing & the Stock Market

More terms: Evaluation and Metrics

• validation set
• precision
• recall
• F1 score
• confusion matrix
• cross-validation