Glossary

Interpretability

Written with AI from the sources below and checked against our facts sheet. Last checked 7 Nov 2026. How this site is made · Spot an error?

Part of: NIST AI Risk Management Framework

Interpretability is the degree to which people can understand what an AI output means and why it was produced, given the system's purpose. NIST separates it from explainability (how the system works) and transparency (what happened). Some models, such as scorecards, are interpretable by design; complex models need explanation tools.

In plain English

Take a weather forecast that says "70% chance of rain." The meteorologist's model can show the pressure systems behind the number: that is an explanation. Knowing that 70% means you should carry an umbrella to your afternoon meeting: that is interpretation.

The NIST AI RMF draws the line in three questions. Transparency answers "what happened." Explainability answers "how" a decision was made. Interpretability answers "why" it was made, and what it means to the user.

In machine learning, the word also describes models. An interpretable model is one a person can read directly, such as a points-based scorecard. A black-box model needs separate tools to explain it.

Example

Fenwick Credit Union, a fictional US lender, scores small-business loan applications. The model returns a risk score of 0.62 for one applicant.

The raw output: "0.62." On its own, it tells a loan officer almost nothing.
An explanation: SHAP values show that a high debt-to-income ratio added the most risk.
An interpretation for the officer: scores above 0.55 fall in the "refer" band, so the officer must verify income documents before deciding.
An interpretation for the applicant: plain-language principal reasons for any denial, which US credit law requires.

Fenwick chose a scorecard over a more complex model for these loans. Officers must justify each decision, and the accuracy gap in testing was small. The trade-off was approved by its AI committee and documented.

Why it matters

NIST. "Explainable and interpretable" is one of the AI RMF's seven characteristics of trustworthy AI. Section 3.5 defines interpretability as the meaning of an AI system's output in the context of its designed functional purpose. Section 3 notes possible trade-offs between predictive accuracy and interpretability.

EU AI Act. High-risk systems must be transparent enough "to enable deployers to interpret a system's output and use it appropriately" (Art. 13(1)). Instructions for use must include information that helps deployers interpret the output (Art. 13(3)(b)(vii)). Human overseers must be able to interpret outputs correctly (Art. 14(4)(c)). Affected people can ask for an explanation of significant decisions (Art. 86).

Testing. The BoK lists interpretability testing among seven test types: can the people who need to understand the outputs actually do so?

How the AIGP exam tests it

The three-way distinction is taught in lesson I.A.4. The tools and tests appear in lesson III.B.7.

Common traps:

Near-synonyms. Transparency is what happened and that AI is in use. Explainability is how. Interpretability is why, and what it means. The exam offers the neighbor as a distractor.
Wrong test. "Claims handlers can't tell why the model flagged a claim" calls for interpretability testing, not bias or performance testing.
Engineering dumps. Publishing source code or hyperparameters does not make outputs interpretable to affected people.
Maximize accuracy. The most accurate model is not automatically right for consequential decisions.

The tell: if the question is about what an output means for the person using it, the answer is interpretability. See also explainable AI.

Frequently asked questions

What is the difference between interpretability and explainability?

In the NIST AI RMF, explainability describes the mechanisms behind how a system works and reached a decision. Interpretability describes what an output means in the context of the system's purpose, and why it matters to the user. Feature weights are an explanation. Telling a loan officer what a score of 0.62 means for this applicant is interpretation.

What is an interpretable model?

A model whose decision logic a person can follow directly. Examples are a credit scorecard that adds points for each factor, or a small decision tree. No separate explanation tool is needed. Complex models such as deep neural networks are not interpretable this way, so teams add post hoc methods like SHAP or counterfactual explanations.

Does the EU AI Act require interpretability?

For high-risk systems, in effect yes. Art. 13(1) requires them to be transparent enough for deployers to interpret their output and use it appropriately. Art. 14(4)(c) requires that human overseers can correctly interpret the output. The Act does not prescribe a technique or require an interpretable model.

Is there a trade-off between accuracy and interpretability?

Sometimes. The NIST AI RMF notes organizations may face a trade-off between predictive accuracy and interpretability. A simple scorecard is easy to interpret but may be less accurate than a complex model. Governance treats this as a decision matched to the stakes, made cross-functionally and documented, not a default.

Sources

Spot an error? Tell us