Glossary
Differential privacy
Written with AI from the sources below and checked against our facts sheet. Last checked . How this site is made · Spot an error?
Part of: AIGP certification and exam
Differential privacy is a mathematical privacy guarantee. The result of an analysis, or a trained model, should be nearly the same with or without any one person's data. It is usually achieved by adding calibrated random noise. A privacy parameter, epsilon, sets the trade-off: smaller epsilon means stronger privacy and less accuracy.
In plain English
Picture a survey on a sensitive topic. Before answering, each person flips a coin in private. Heads, they answer truthfully. Tails, they answer yes or no at random. No single answer can be held against anyone, yet the overall rate can still be estimated. This old survey trick is called randomized response.
Differential privacy turns that intuition into a measurable guarantee. Its promise, in NIST's words, is that "the chance of an outcome is about the same" with or without any one person's data. An attacker learns roughly the same about you whether you joined the dataset or not.
The privacy parameter, epsilon, sets how close "about the same" must be. Smaller epsilon means more noise, stronger privacy and less accurate results. Each new release spends part of the privacy budget, so losses add up over time.
Example
The Port Ellery Transit Authority, a fictional city agency, wants to publish ridership by stop and hour. It also wants to share trip data with an AI route-planning vendor. Some late-night stops have only one or two regular riders, so exact counts could reveal when a specific person travels.
The agency:
- adds calibrated noise to each published count, so no rider's presence changes the totals noticeably;
- protects whole riders, not single trips, because NIST notes trip-level protection can still expose someone's regular destinations;
- sets an annual privacy budget and tracks how much each release spends;
- gives the vendor differentially private synthetic trip data instead of raw card taps;
- checks that planners can still spot the busiest routes.
Counts for quiet stops become less precise. The agency accepts that, because route planning needs patterns, not individual journeys.
Why it matters
- NIST. The AI RMF lists "privacy-enhanced" among its seven trustworthy-AI characteristics. It names privacy-enhancing technologies as support and warns they can cost accuracy when data is sparse. SP 800-226 (March 2025) is NIST's guide to evaluating differential privacy claims.
- Training models. NIST warns that neural networks are particularly prone to memorizing training data. Differentially private training limits what attacks such as membership inference can learn about any one person.
- GDPR. Differential privacy can support data minimization (Art. 5(1)(c)) and data protection by design (Art. 25) under the GDPR. Whether outputs count as anonymous depends on the parameters and context.
- Real deployments. NIST notes that the U.S. Census Bureau and large technology companies run differential privacy in production.
How the AIGP exam tests it
Lesson III.B.5 teaches each privacy-enhancing technique as having one job. For differential privacy, that job is noise.
- The tell. Publish statistics, or train a model, so no individual can be singled out while overall patterns survive.
- Trap 1: "no cost to accuracy." False. The noise that protects people reduces precision. Stronger privacy means more noise.
- Trap 2: confusing the neighbors. Data stays at its source: federated learning. New artificial records: synthetic data. Identifiers swapped for a key: pseudonymization.
- Trap 3: the wrong threat. Differential privacy protects individuals in outputs. It does not stop data poisoning, and it does not make unlawful pooling of raw records lawful.
Frequently asked questions
What is epsilon in differential privacy?
Epsilon is the privacy loss parameter. It bounds how much any one person's data can change the result. A smaller epsilon means results must look more alike with or without that person, which needs more noise: stronger privacy, lower accuracy. Epsilon adds up across releases, so organizations track a total privacy budget.
Does differential privacy reduce accuracy?
Yes. The random noise that hides individuals also blurs results, especially for small groups and rare cases. NIST calls this the privacy-utility trade-off. Large datasets and aggregate questions suffer least. Teams choose epsilon by weighing the harm of disclosure against the accuracy the use case needs.
Is differentially private data anonymous under the GDPR?
Not automatically. The GDPR asks whether anyone can reasonably identify a person (Recital 26). Differential privacy with a small epsilon makes that much harder. But NIST notes that large epsilon values can still leave outliers exposed. Anonymity is judged case by case, so document the parameters and the reasoning.
What is the difference between differential privacy and federated learning?
Differential privacy limits what outputs reveal about any individual by adding noise. Federated learning keeps raw data where it lives and moves only model updates. They solve different problems and are often combined, because model updates in federated learning can leak information unless noise is added to them.