Confidence scores express the system's estimated reliability for a given output, typically as a value between zero and one. They may derive from model probabilities, retrieval scores, or agreement across methods.
Scores are not probabilities of correctness and require calibration to be interpretable.
Alternative Names:
Confidence Level, Model Confidence
Why it Matters?
Confidence scores are useful for triage and dangerous as assurance. A well-calibrated score lets a workflow route low-confidence extractions for review while accepting high-confidence ones, which is how volume gets handled. But language models are frequently confident and wrong, so a high score is not verification. The question to ask a vendor is whether their scores have been calibrated against measured accuracy or merely reflect internal model probabilities.
Frequently Confused with
Related terms
Frequently asked questions
Does a high confidence score mean the output is correct?
What makes a confidence score useful?
