Inference is the operational phase in which a model processes an input and generates a response. It consumes computation proportional to input and output length and is what users experience as the system responding.
Training builds the model; inference uses it.
Alternative Names:
Model Inference, Prediction
Why it Matters?
Inference cost scales with the volume of text processed, which is why systems that send entire document sets to a model for every question are expensive and slow compared to retrieval architectures that supply only relevant passages. For legal buyers this shows up as pricing tied to document volume or usage, and it is the economic reason retrieval design affects both accuracy and cost.
Frequently Confused with
Related terms
Frequently asked questions
What is the difference between training and inference?
Why does inference cost matter?
