Data provenance vs data lineage

Side by side

When each applies

Provenance answers questions about origin and rights. W3C's PROV specifications describe provenance as the record of the entities, activities and people involved in producing data. That record lets others judge its quality and trustworthiness. For AI, provenance answers questions like these. Was this personal data collected under a legal basis that covers training? Does the license allow commercial model training? Was the scraped content subject to a text and data mining opt-out?

Lineage answers questions about the path. Which records were dropped as duplicates? Who changed the labels, and when? Which version of the dataset trained model 3.2? Lineage lets a team rebuild the exact training set and trace a fault to its source.

Both run through the life cycle. Neither is a one-time form. New sources need provenance records when added, and every pipeline run should log lineage. The NIST AI RMF notes that keeping the provenance of training data supports both transparency and accountability.

The law asks for both. For high-risk systems, Art. 10(2) of the EU AI Act requires data governance covering collection processes and data origin, including the original purpose of personal data. It also covers preparation steps such as annotation, labeling and cleaning. GDPR rights requests also depend on knowing which datasets and models contain someone's data.

Worked example

Medrin Health, a fictional hospital network, trains a model to predict hospital readmissions. Six months after launch, an audit finds the model underestimates risk for rural patients.

Provenance shows the training data came from three hospitals under a data-sharing agreement that covers service improvement. The rights check holds. But it also shows that only one of the three hospitals serves rural areas, a representativeness warning nobody acted on.

Lineage pinpoints the bigger cause. A deduplication step matched patients on address formats. It treated many rural records, which shared postal boxes, as duplicates and deleted them. The rural share of the data fell from 18% to 7% between source and model.

Medrin fixes the matching rule, rebuilds the dataset from the versioned source and retrains. Because every step was logged, the investigation took days rather than months. Medrin updates its datasheet and model card to describe the fix. It also adds a check that compares each group's share of the data before and after every pipeline step.

The exam trap

The classic trap is swapping the two terms. Origin, rights, license and consent point to provenance. Transformation steps, versions and the path to the model point to lineage.

The tells:

• "Where did it come from, and may we use it?" is provenance.
• "What happened to it on the way?" is lineage.
• Investigating data poisoning or a skewed dataset usually needs lineage.
• Answering a copyright claim or a consent withdrawal usually needs provenance.
• "Document it later" is wrong. The BoK says establish and document both from the start.

A datasheet is the human-readable summary of both records. Practice in lesson III.B.2 and the documentation lesson III.B.6.