Data lineage

In plain English

Think of a parcel tracking page. It shows every depot the package passed through, when it was scanned and who handled it. Data lineage is the tracking page for training data.

It records each step between the original source and the model: extracts, joins, filters, deduplication, labeling, aggregation and every dataset version. With good lineage you can rebuild the exact dataset that trained any model version.

Lineage is usually paired with data provenance, but the two answer different questions. Provenance is the origin story: who collected the data, when, and under what license or consent. Lineage is the journey that follows. Our data provenance vs data lineage comparison sets them side by side.

Example

Brindle Analytics, a fictional lender-services firm, notices that its small-business credit model scores farm businesses unusually low. Its lineage records let the team replay the pipeline:

1. Raw applications from three partner banks, saved as dataset version 1.0.
2. A join with company registry data, which failed to match many farms.
3. A filter that dropped rows with missing registry fields.
4. Default labels and monthly aggregation, saved as version 1.3.

Step 3 removed most farm applicants before training. Without lineage, the team might have retuned the model and never found the cause. Instead it repairs the join, rebuilds the data as version 1.4, retrains and re-validates. The model card and technical documentation now link model 2.1 to dataset 1.4.

Why it matters

• EU AI Act. For high-risk systems, data governance must cover "relevant data-preparation processing operations," such as annotation, labeling, cleaning, updating, enrichment and aggregation (Art. 10(2)(c)). Technical documentation must describe how data was obtained and selected, labeling procedures and cleaning methods (Annex IV, point 2(d)). Lineage records are how providers show this.
• Bias. Art. 10(2)(f) and (g) require providers to examine likely biases and take measures against them. Lineage shows which step introduced a skew.
• Security. Lineage helps trace data poisoning back to the batch that introduced it.
• Privacy rights. To act on an erasure request under GDPR Art. 17, you must know which datasets and models contain the person's data.
• NIST. The AI RMF says maintaining the provenance of training data supports transparency and accountability (section 3.4). Lineage extends that record through every transformation.

How the AIGP exam tests it

Lesson III.B.2 teaches lineage and provenance as a pair, and the exam tests the difference.

• The tell. A question about a transformation is about lineage: a join, a filter, an aggregation step, or which dataset version trained which model. A question about origin, license or consent is about provenance.
• The BEST answer pattern. "How can the organization BEST ensure training data is accurate, representative and unbiased?" Track lineage and provenance.
• Diagnose first. When a model underperforms for a group, rebuilding lineage comes before synthetic top-ups or threshold changes.
• Trap: purchased data needs no lineage. Vendor promises are claims, not evidence. Bought data gets the same scrutiny as any other source.