Explainability requirements demand that outputs be traceable to their basis. In document-grounded systems this means citing the specific source passage supporting each assertion rather than explaining the model's internal reasoning.
True mechanistic interpretability of large models remains limited.
Alternative Names:
Explainability Requirement, Output Explanation
Why it Matters?
Source citation is the form of explainability that actually works for legal applications, because it permits verification against the underlying document regardless of what happened inside the model. Requiring that every assertion carry a page-level citation converts an unverifiable output into a checkable one. Systems that explain their reasoning in prose without citing sources provide the appearance of explainability without the verification.
Frequently Confused with
Related terms
Frequently asked questions
What form of explainability works for legal use?
Why is prose reasoning insufficient?
