Glossary

SHAP

Written with AI from the sources below and checked against our facts sheet. Last checked 7 Nov 2026. How this site is made · Spot an error?

Part of: AIGP certification and exam

SHAP (SHapley Additive exPlanations) is a method for explaining individual predictions of a machine learning model. It gives each input feature a value showing how much it pushed one prediction up or down from the model's average output. The values come from Shapley values in game theory and add up exactly to the prediction.

In plain English

Four friends share a taxi home, each dropped off at a different place. How should they split the fare fairly? Game theorist Lloyd Shapley answered this kind of question in 1953. Each person pays their average extra cost across every possible order of drop-offs.

SHAP applies the same fairness math to a model's inputs. The "fare" is the gap between one prediction and the model's average prediction. Each feature gets its fair share of that gap, positive or negative.

SHAP was introduced by Lundberg and Lee in 2017. It is a local method: it explains one prediction. Averaging the size of SHAP values over many predictions gives a global picture of which features matter most.

Example

Ferncliff Mutual, a fictional insurer, uses a model to flag home policies for a price review at renewal. The model's average risk score is 40. For one homeowner, Dana, it outputs 70. SHAP splits the 30-point gap:

Feature	SHAP value
Two water-damage claims in three years	+18
Roof age of 24 years	+9
Distance to the nearest fire station	+6
Monitored alarm installed	-3
Total	+30

The values add up exactly: 40 plus 30 is 70. Reviewers can see why Dana's policy was flagged, and Dana can contest the main reason if her claims history is wrong.

Across all customers, the team averages SHAP values and finds that ZIP code ranks high. Location can act as a proxy for protected traits, so the feature goes to a fairness review.

Why it matters

GDPR. People are owed meaningful information about the logic of solely automated decisions (Art. 15(1)(h)). In Dun & Bradstreet Austria (C-203/22, 2025), the EU's top court said this means explaining the procedure and principles actually applied. SHAP can identify the main factors, but they still need translating into plain language.
EU AI Act. High-risk systems must let deployers interpret outputs (Art. 13) and let oversight staff interpret them correctly (Art. 14(4)(c)). Art. 86 gives affected people a right to an explanation of the main elements of certain decisions (EUR-Lex).
US. Lenders must give specific reasons for adverse credit decisions under ECOA and Regulation B. Colorado's SB 26-189 requires an explanation within 30 days of an adverse consequential decision, from 1 Jan 2027.
NIST. MEASURE 2.9 expects the model to be explained, validated and documented, with outputs interpreted in context.

How the AIGP exam tests it

SHAP sits in the explainability toolkit of lesson III.B.7. You need to know what it answers, not the math.

The tell. "Why was this person declined?" or "which inputs drove this prediction?" SHAP fits.
Trap 1: "what would I need to change?" That calls for a counterfactual explanation. SHAP says why; a counterfactual says what would flip the result.
Trap 2: global vs local. Global feature importance describes the model on average. SHAP explains one prediction, though it can be averaged into a global view.
Trap 3: SHAP as proof. SHAP describes what the model did, not what causes outcomes in the real world. Slack and colleagues built biased models that produced innocent-looking SHAP and LIME explanations. Explanations need testing like any other control. See explainable AI for the wider toolkit.

Frequently asked questions

What do SHAP values mean?

A SHAP value is one feature's contribution to one prediction, measured against the model's average output. A positive value pushed the prediction up, and a negative value pushed it down. The size shows how much. The average output plus all the SHAP values for a prediction add up exactly to that prediction.

What is the difference between SHAP and LIME?

Both explain individual predictions of complex models. SHAP uses Shapley values from game theory and comes with guarantees, such as values that add up to the prediction. LIME fits a simple stand-in model around one prediction, and its results can vary between runs. SHAP can be slower to compute.

Is SHAP local or global?

SHAP is local: it explains one prediction at a time. Because the values are consistent across predictions, you can average their absolute size over a whole dataset. That produces a global ranking of which features matter most to the model overall.

Can SHAP prove a model is fair?

No. SHAP shows which inputs drove a model's outputs, which can reveal a proxy for a protected trait. But fairness needs outcome testing across groups with fairness metrics chosen in advance. Researchers have also shown that a deceptive model can produce innocent-looking SHAP explanations, so treat them as evidence to check, not proof.

Sources

Spot an error? Tell us