Govern the designing and building of the AI system.
- ✓Plan in the IAPP order: problem, use case, laws, gaps and risks, then data.
- ✓Make the business case (why AI, costs and benefits) and check feasibility (technical, data, organizational readiness).
- ✓A good use case names purpose, users, affected people, setting, success measures and boundaries.
- ✓Documented approved uses are your defense against scope creep.
- ✓By-design activities belong in planning and design, never after deployment.
- IAPP planning order:
- Business problem → use case → applicable laws → gaps and risks → data needed.
- Use case:
- A specific description of what an AI system will do, for whom, in what setting, and within what boundaries.
- Baseline:
- How the task performs today without the AI, used to judge whether the system is actually an improvement.
- Scope creep (function creep):
- Gradual expansion of a system into uses that were never assessed or approved.
- MAP function (NIST AI RMF):
- The NIST function that establishes context: intended purpose, settings, laws, norms and potential impacts.
- Business case:
- The argument for an AI project: the problem, why AI beats simpler options, measurable objectives, and direct and indirect costs weighed against benefits.
Exam tips: Ordering questions: business problem → use case → applicable laws → gaps and risks → data. Any option that starts with data, vendors or models is a trap. · Watch FIRST vs. NEXT. If the stem says the problem and use case are already defined, the answer is 'identify applicable laws', not 'determine the business problem'. · Laws attach to the use and the jurisdiction, never to the tech stack or the cloud provider. · 'A simpler, non-AI solution fits better' can be the BEST answer. A business case asks 'why AI?', and a failed feasibility check (technical, data, organizational readiness) is a valid reason to stop.
- ✓Perform or review an impact assessment before building; reassess after significant changes, after incidents and on a schedule.
- ✓Its job is to locate the greatest risks so mitigation is targeted.
- ✓AI impact assessment, DPIA and FRIA overlap but have different scopes and owners.
- ✓Good documentation covers stakeholders, affected demographics, technical choices, harms and mitigations.
- AI impact assessment:
- A structured review, done before build and revisited over time, of how an AI system could affect people, groups and society.
- DPIA:
- GDPR's data protection impact assessment, required of controllers before personal-data processing likely to result in high risk.
- FRIA:
- The EU AI Act's fundamental rights impact assessment, a duty on certain deployers of high-risk systems before first use.
- ISO/IEC 42005:
- International guidance (2025) on how to conduct AI system impact assessments across the life cycle.
Exam tips: Timing tell: impact assessments happen BEFORE build and deployment. 'After launch, once we have real data' is the wrong-stage trap. · 'The DPIA already covers it' is synonym bait. Privacy is one slice of AI impact. · FRIA = deployer. A vendor's conformity assessment never discharges the deployer's own FRIA. · LEAST-necessary questions: stakeholder concerns, demographics and algorithm details stay. Budgets and marketing plans go.
- ✓Turn AI principles into concrete requirements during design.
- ✓Requirements cover what the system does and how it must behave.
- ✓Model selection weighs explainability, complexity, control and security, and the rationale is recorded.
- ✓Check data availability, representativeness and proxies before committing to a design.
- ✓Diverse teams catch biased assumptions early.
- Ethics by design:
- Building ethical requirements into an AI system from the plan-and-design phase rather than reviewing them at launch.
- Requirements gathering:
- Collecting what a system must do and how it must behave, including legal, fairness, privacy and explainability needs.
- Interpretable model:
- A model whose decision logic people can follow directly, such as a scorecard or a small decision tree.
- Proxy variable:
- A seemingly neutral input, like postcode, that stands in for a protected trait, like ethnicity.
- Build, buy or adapt:
- The choice between training your own model, buying a vendor system and fine-tuning an existing model.
Exam tips: Design-stage bias question? A diverse, cross-functional team beats 'collect more data' and 'buy a fairness toolkit'. · Accuracy vs. explainability: the BEST answer weighs the trade-off with evidence and documents it. It doesn't let one metric decide. · 'Purpose of AI' includes asking whether AI is needed at all. · Any ethics-by-design step scheduled 'at deployment' or 'after an incident' is the wrong-stage trap.
- ✓In, on, out of the loop: pick by severity, reversibility and volume. Human-in-command sits above all three.
- ✓Automation bias can hollow out human review, so design against it.
- ✓Set performance, fairness, confidence and action thresholds before building.
- ✓Feedback channels and operational controls (override, stop, fallback, escalation, logging) are designed in, not added later.
- Human-in-the-loop:
- A person reviews or approves each AI output before it takes effect.
- Human-on-the-loop:
- The AI acts on its own while a person monitors and can intervene or override.
- Human-out-of-the-loop:
- The AI operates fully automatically with no real-time human involvement.
- Automation bias:
- The tendency to over-rely on automated outputs, even when they are wrong.
- Operational controls:
- Mechanisms built into a running system, such as overrides, stop functions, fallbacks, escalation thresholds and logs, that let humans steer it.
- Human-in-command:
- Oversight in which people decide whether, when and how an AI system is used at all, and can overrule or switch it off.
Exam tips: 'By design' in the stem points to controls specified during design: overrides, thresholds, feedback channels. 'Sort out oversight after launch' is the trap. · An annual or periodic audit is NOT an operational control. It's assurance after the fact. · A human who approves everything isn't oversight. Look for designs that force independent judgment, like rating before seeing the AI's answer. · Match oversight to severity and reversibility. 'Remove humans because the model beats the benchmark' is an absolute-abdication trap.
- ✓Risks and contributing factors come from inside and outside the organization.
- ✓Probability × severity ranks harms against risk tolerance; AI severity includes scale and reversibility. Respond by corner: act, mitigate, plan for contingencies, or review.
- ✓Prohibitive, major, moderate: unacceptable risk means stop safely until it's managed.
- ✓Stakeholder mapping finds the high-impact, low-influence groups.
- ✓Mitigation hierarchy: avoid, reduce, restore, compensate, in that order of preference.
- Harms matrix:
- A probability × severity grid used to rank potential harms so the worst get attention first.
- Risk tolerance:
- The level of risk an organization is willing to accept in pursuit of its goals.
- Risk mitigation hierarchy:
- An ordered preference for responses: avoid the risk, then reduce it, then restore, and compensate as a last resort.
- Stakeholder mapping:
- Identifying everyone affected by or influencing a system, including people who never use it.
- Residual risk:
- The risk that remains after mitigations are applied, which must be accepted, transferred or further treated.
- Contributing factor:
- A condition, such as a rushed timeline or skills gap, that makes harm more likely or more severe.
Exam tips: The matrix RANKS, the hierarchy picks the RESPONSE, the stakeholder map shows WHO is affected (including people who never touch the system). If the stem asks how to respond to ranked risks, the answer is the hierarchy. · HIGHEST in the hierarchy = avoid or eliminate at source. Disclaimers, warnings, insurance and compensation sit low. · Rare but catastrophic → contingency plan, never 'ignore'. Unacceptable (prohibitive) risk → NIST says cease development or deployment in a safe manner until it's managed. · Quantifying risk never makes an organization 'immune', and AI risks belong in the same enterprise risk register as cyber and privacy risks.
- ✓Use-case evaluation asks whether to proceed, and repeats whenever the use changes.
- ✓Benchmark against a baseline and break results down by group.
- ✓Pilots are limited, criteria-driven trials with real users and an exit plan.
- ✓Document purpose, requirements, assessments, trade-offs, oversight and evidence as you go.
- ✓Documentation establishes compliance, manages risk and preserves accountability.
- Use-case evaluation:
- Assessing whether a specific use of AI is lawful, beneficial and within risk tolerance before proceeding, and again whenever the use changes.
- Benchmarking:
- Comparing a system's performance offline against a reference such as the current process, another model or a standard test set.
- Pre-deployment pilot:
- A limited, controlled trial with real users and pre-agreed success criteria before the go/no-go decision.
- Shadow mode:
- Running an AI alongside the existing process so its outputs can be compared without driving real decisions.
- Technical documentation (EU AI Act):
- The record a high-risk system's provider must prepare before placing it on the market and keep up to date.
Exam tips: Real users + realistic conditions + before go/no-go = pilot. Offline comparison = benchmarking. Live production data = post-deployment. · Same model, new use → new evaluation and assessment. 'Just update the model card' is too narrow. · Documentation questions reward the option that captures the WHY (rationale, alternatives, trade-offs), recorded at the time. · A benchmark win never 'proves' a system safe or 'permanently' settles anything.
- ✓HUDERIA, EU AI Act Art. 9(5), ISO/IEC Guide 51 and the hierarchy of controls all rank designing the risk out first and warning, remedying or accepting last.
- ✓NIST's avoid/mitigate/transfer/accept and the four Ts are menus of the same options; transfer never protects the affected person.
- ✓Incident orders vary, but all prepare, contain early, preserve evidence, notify on time and learn last.
- ✓Performance testing = speed and scale; validation testing = meets requirements, including accuracy.
- EU AI Act Art. 9(5) order:
- For high-risk providers: eliminate or reduce risks by design, then add mitigation and control measures, then give deployers information and, where appropriate, training.
- ISO/IEC Guide 51 three-step method:
- The safety-standards order for reducing risk: inherently safe design, then protective measures, then information for use.
- Four Ts:
- A business risk-response menu: treat, transfer, tolerate or terminate.
- Risk transfer:
- Shifting a risk's financial consequences to another party, such as an insurer or contractor; it doesn't reduce the harm to affected people.
- Performance testing:
- Testing whether a system meets speed and scale targets, such as latency, throughput and resource use under load.
Exam tips: Different ladders, same direction: design it out → control it → warn, train, remedy or accept. Pick the option on the highest tier, whichever vocabulary it uses. · Transfer (insurance, contracts) moves cost, not harm. It is never the BEST answer for protecting the people affected. · Incident lists differ, but containment comes early when harm is ongoing, evidence is preserved, notification runs in parallel and lessons come last. An early 'communicate' step means internal escalation. · 'Performance testing' on the exam usually means speed and scale (latency, throughput, load). Accuracy, precision and recall against requirements belong to validation.
- ✓Engagement process: salience, diversity, positionality, level, methods, AI actors, communication plans.
- ✓Match engagement depth to impact: inform, consult, involve or collaborate, empower.
- ✓Optionality keeps alternatives, including non-AI ones, genuinely available.
- ✓Redress means people can contest, reach a human who can change the outcome, and get things put right.
- Stakeholder salience:
- How much weight a stakeholder's voice should get, judged by power, legitimacy, urgency and how heavily the system affects them.
- Positionality exercise:
- A structured reflection in which a team states how its members' backgrounds and roles shape what they notice, to surface blind spots before design choices.
- Optionality:
- Keeping feasible alternatives open, including non-AI options and opt-outs to a human route, and avoiding lock-in.
- Redress:
- The means for people to report, contest and get correction of an AI system's harmful or wrong outcomes, designed in from the start.
Exam tips: Salience isn't just power: a low-influence group facing severe impact ranks high. Options that only engage the vendor, executives or business partners are traps. · Positionality = how the team's own backgrounds shape what it sees. The fix is diverse voices and stated vantage points, not 'more data'. · Optionality includes a non-AI option (NIST MANAGE 2.1) and, where feasible, an opt-out to a human route with equal service. · Redress is designed in: contest, human review with power to change the outcome, correction, the ability to halt. Disclaimers or compensation alone aren't redress.
Govern the collection and use of data in training and testing the AI model and system.
- ✓Assess and document lawful rights to every source before training.
- ✓Check quality, quantity, integrity and fitness for purpose as separate dimensions.
- ✓Fit for purpose includes representing the population the system will serve.
- ✓Rights and origin before technical work; cleaning before splitting.
- ✓Data owners decide how data may be used; data stewards keep it in good order. Lawful still has to pass an ethics check.
- Lawful rights to data:
- The documented legal basis, consent, license or contract that permits an organization to collect and use a dataset for a specific purpose.
- Purpose limitation:
- The principle that data collected for one purpose shouldn't be reused for an incompatible one.
- Data integrity:
- Assurance that data has stayed complete and unaltered, protected from corruption and tampering.
- Fitness for purpose:
- Whether data suits the specific task and population a model will serve, beyond being large and accurate.
- Data minimization:
- Using only the personal data a purpose actually needs.
- Data owner vs. data steward:
- The owner is the accountable leader who decides how a dataset may be used; the steward manages its quality, documentation and day-to-day handling.
Exam tips: Lawful rights come FIRST. Options that start cleaning, deduplicating or modeling before rights are checked are premature. · 'Publicly available' isn't 'licensed for training', and 'anonymize it and the rights problem disappears' is a trap. · If the stem concedes the data is big and accurate but a reviewer still objects, the answer is fitness for purpose. · Cross-border data: review origin, destination and controlling law BEFORE any processing.
- ✓Provenance is origin; lineage is the journey to the model.
- ✓Both make accuracy, representativeness and bias checkable.
- ✓They power incident forensics, rights requests and reproducibility.
- ✓Use source records, transformation logs, dataset versioning and access logs, automated where possible.
- Data provenance:
- The origin of data: who created or collected it, when, and under what authority, consent or license.
- Data lineage:
- The traceable path of data from its source through every transformation into a model.
- Dataset versioning:
- Keeping identifiable versions of datasets so each model can be linked to the exact data it used.
- Chain of custody:
- An unbroken record of who handled something and what they did, borrowed from evidence handling.
Exam tips: Provenance = where it came from (origin, authority, license). Lineage = what happened to it (transformations). Swapped definitions are a classic distractor. · 'BEST way to ensure training data is accurate, representative and unbiased' → track lineage and provenance. · Never 'assume purchased data is accurate'. Vendor promises are not evidence. · Diagnose with lineage before fixing. Synthetic top-ups or threshold tweaks before understanding the cause are premature.
- ✓Train on training data, tune on validation data, judge once on test data.
- ✓Watch for overfitting, underfitting, leakage and contamination.
- ✓Seven test types: unit, integration, validation (meets requirements), performance (speed and scale), security, bias, interpretability.
- ✓Automate for coverage, test edge cases and malicious inputs, add human interpretation, and choose metrics that match the harm.
- Validation set:
- Data held back from training and used during development to tune settings and compare model versions.
- Test set:
- Data kept completely separate and used once, at the end, to estimate real-world performance.
- Overfitting:
- When a model memorizes its training data and performs poorly on new data.
- Underfitting:
- When a model is too simple or undertrained to capture the pattern, performing poorly everywhere.
- Data leakage:
- When a training feature carries information about the outcome that won't be available at prediction time, inflating scores.
- Precision and recall:
- Precision is the share of flagged cases that are truly positive; recall is the share of true positives the model catches.
Exam tips: Training and test data must stay separate. Options that evaluate on training data, or tune against the test set, are traps. · Overfitting = great on training, poor on new data. Underfitting = poor everywhere. · Full user-base coverage → automated testing. Blind trust in automated metrics → automation bias. The BEST evaluation adds human interpretation. · Bias test fails before release → diagnose, remediate, re-test, document. 'Ship and monitor' and 'just delete the protected attribute' are traps.
- ✓NIST groups bias as systemic, statistical/computational and human.
- ✓Know the causes: sampling, historical, measurement, temporal, computational, confirmation.
- ✓One dataset can carry several biases, each needing its own fix.
- ✓Examine, pick metrics, mitigate, re-test, document, then keep watching.
- Sampling bias:
- Bias from a training sample that doesn't represent the population the system will serve.
- Historical bias:
- Bias from data that accurately records a past shaped by unfair treatment.
- Measurement bias:
- Bias from using a feature or label that is a poor or uneven proxy for what you actually care about.
- Temporal bias:
- Bias from data that is unbalanced across time, so it no longer reflects current reality.
- Systemic bias:
- Bias embedded in institutions and society that flows into data even when each step looks neutral.
- Confirmation bias:
- The human tendency to favor information that supports existing beliefs.
Exam tips: Bias questions hinge on the CAUSE: who was sampled (sampling), when (temporal), what was measured (measurement), an unfair past (historical), an external skew (computational). · 'Remove the protected attribute' is rarely the BEST answer: proxies carry the signal. · 'More data' only helps if it's the RIGHT data. More of a skewed sample or an outdated era makes things worse. · Temporal bias fix = recent, representative data (reweight or restrict the window), not more history.
- ✓Data poisoning corrupts training data; defend the pipeline with lineage, access control and integrity checks.
- ✓Models can memorize and leak personal data, so minimize first.
- ✓Differential privacy adds noise; federated learning keeps data home; synthetic data fills gaps; homomorphic encryption, SMPC and TEEs protect data while it's being computed on.
- ✓Anonymization is irreversible; pseudonymization isn't.
- Data poisoning:
- An attack that inserts corrupted or manipulated data into a training set so the model learns harmful or wrong behavior.
- Backdoor:
- A hidden behavior planted through poisoning that activates only when a specific trigger appears.
- Differential privacy:
- Adding calibrated statistical noise so individuals can't be singled out while overall patterns survive.
- Federated learning:
- Training a model across separate data holders so raw data stays put and only model updates are shared.
- Synthetic data:
- Artificially generated records that mimic real data, used to fill gaps, cover rare cases and improve accuracy.
- Pseudonymization:
- Replacing identifiers with codes that a separately held key can reverse; still personal data for whoever can re-link it.
Exam tips: Map each PET to ONE job: noise (differential privacy), train in place (federated learning), generate new records (synthetic), strip identifiers (anonymization), swap for a key (pseudonymization), compute on encrypted data (homomorphic encryption), joint result without sharing inputs (SMPC), hardware enclave (TEE). 'Interchangeable' is the trap. · Poisoning = training time; evasion = inference time. Poisoning defenses are data-governance controls: lineage, access control, integrity checks. · Raw data legally can't move → federated learning. Real examples too scarce → synthetic data. · Pseudonymized isn't anonymized. If re-linking is possible, it's still personal data.
- ✓Documentation lets others validate results, establishes compliance and manages risk.
- ✓Datasheets describe datasets in seven sections, from motivation to maintenance.
- ✓Model-card inputs (data, metrics, factors, per-group results) are captured during testing.
- ✓Test reports record versions, methods, thresholds, failures, fixes and sign-off.
- Datasheet for a dataset:
- A standard document describing a dataset's motivation, composition, collection, processing, uses, distribution and maintenance.
- Model card:
- A short summary of a trained model's purpose, performance (including by group), limitations and intended use.
- Test report:
- A dated record of what was tested, how, against which criteria, the results including failures, remediation and sign-off.
- Reproducibility:
- The ability for others to obtain consistent results by repeating the documented process.
Exam tips: Datasheet = the DATASET (motivation, composition, collection…). Model card = the MODEL (purpose, performance, limits). Don't swap them. · Records must include failures and remediation, not just the final passing score. · Thresholds are documented BEFORE testing. Anything set after seeing results is goalpost-moving. · The model card is finalized at release (III.C), but its inputs come from training and testing records.
- ✓TEVV runs across the life cycle: verification checks the spec, validation checks the real-world fit.
- ✓False positives are false alarms, false negatives are misses; context decides which hurts more.
- ✓Accuracy hides rare-event failures; precision, recall and F1 tell the truth.
- ✓Fairness metrics (parity, equal opportunity, calibration) can conflict, so choose and document one up front.
- ✓Explain globally and locally (SHAP, LIME, counterfactuals), and check calibration and repeatability so confidence means something.
- TEVV:
- Test, evaluation, verification and validation: NIST's umbrella term for the checks run across the AI life cycle.
- Verification vs. validation:
- Verification confirms a system meets its specification (built it right); validation confirms it fulfills the requirements for its intended use (built the right thing).
- False positive / false negative:
- A false positive flags something that isn't there (a false alarm); a false negative misses something that is.
- F1 score:
- A single score that balances precision and recall; it falls sharply if either one is poor.
- Fairness metrics:
- Yardsticks such as demographic parity, equal opportunity and calibration that compare outcomes across groups; when base rates differ, they can't all be met at once.
- Counterfactual explanation:
- An explanation that states the smallest change to the inputs that would have changed the outcome.
Exam tips: Rare outcome plus a big accuracy number = the base-rate trap. Ask for recall, precision and results by group. · Precision guards against false alarms; recall guards against misses. Pick the metric that matches the costlier error in the scenario. · Verification = built it right (meets the spec). Validation = built the right thing (fits the intended use). Repeatability = same team, same setup; reproducibility = someone else, from your documentation. · No model meets every fairness metric when base rates differ: the BEST answer picks one up front, ties it to the harm and documents why. 'What would I need to change?' calls for a counterfactual explanation.
- ✓Write the test plan first: objectives, scope, edge and malicious cases, criteria, roles.
- ✓Independent validation, peer review and gates make results credible.
- ✓Re-test after every change: regression testing keeps old passes valid.
- ✓Watch for label noise, class imbalance, poor calibration, environment mismatch and feedback loops.
- ✓Layer mitigations and weigh their trade-offs.
- Test plan:
- A document written before testing that sets objectives, scope, scenarios, metrics, pass/fail criteria and roles.
- Regression testing:
- Re-running tests after any change to data, code, features, hyperparameters or thresholds, to confirm earlier results still hold.
- Inter-rater reliability:
- How consistently independent labelers give the same label to the same item; low agreement signals label noise.
- Feedback loop:
- When a model's outputs influence the data it is later trained or tested on, reinforcing its own patterns.
- Defense in depth:
- Layering safeguards at different points in a system so a single failure doesn't cause harm.
Exam tips: Plan before, report after. Pass criteria written after seeing the results are goalpost-moving. · Any change (data, features, code, hyperparameters, thresholds) → regression testing. 'It was only a small tweak' is the trap. · Builders don't validate their own work alone. Look for independence, peer review and formal gates. · Layered mitigations beat one perfect filter, and every layer has trade-offs (over-blocking, delay, cost) that must be weighed and documented.
Govern the release, monitoring and maintenance of the AI system.
- ✓Readiness = the go/no-go gate between building and deploying.
- ✓Check criteria, documentation, conformity, operational readiness and sign-offs.
- ✓Monitoring, human oversight, incident response and rollback must exist on day one, not 'next sprint'.
- ✓Release in stages (canary, shadow mode) with a tested rollback plan.
- ✓Testing continues after launch, on real production data.
- Readiness assessment:
- The formal check after development and before deployment that decides whether a system is ready to release (go/no-go).
- Go/no-go criteria:
- Acceptance thresholds agreed in advance (performance, fairness, robustness, security, documentation) that a system must meet before release.
- Sign-off:
- A named, accountable person's recorded approval that release criteria are met and the residual risk is accepted.
- Shadow deployment:
- Running a new model on live traffic without acting on its outputs, to compare it safely against the current system.
- Canary (phased) release:
- Releasing to a small share of users first and expanding only if monitoring stays healthy.
- Rollback plan:
- A tested way to return quickly to the previous version if the new one misbehaves.
Exam tips: Sequence anchor: the readiness assessment comes AFTER development and BEFORE deployment. Post-deployment (inference) testing on production data comes AFTER deployment. · Any option that lowers or redefines a threshold to hit a launch date is a trap. Criteria are set in advance and misses are escalated. · 'It passed validation, so no more testing is needed' is always wrong. Lab results never guarantee production performance. · Watch for self-approval: the team that built the model shouldn't be the only one signing it off.
- ✓Model card = concise summary: purpose, performance by subgroup, limits, intended and out-of-scope uses.
- ✓Inventory = org-wide register; technical documentation = proof for regulators; datasheet = about a dataset.
- ✓EU high-risk providers: QMS, technical documentation, conformity assessment, declaration, CE marking, registration, then market.
- ✓Substantial modifications trigger a new assessment, so plan your update envelope up front.
- Model card:
- A short, structured summary of one model's purpose, performance (including by subgroup), limitations, and intended and out-of-scope uses.
- Datasheet for datasets:
- A document recording a dataset's motivation, composition, collection process and recommended uses.
- AI system inventory:
- An organization-wide register of AI systems with their owners, documentation, incident plans, data dictionaries and links to code.
- Conformity assessment:
- The pre-market process showing a high-risk AI system meets the EU AI Act's requirements, by internal control or with a notified body.
- CE marking:
- The mark a provider affixes to show a product conforms to applicable EU requirements, after the conformity assessment and declaration.
- Substantial modification:
- A change not foreseen in the initial conformity assessment that affects compliance or the intended purpose; it triggers a new assessment.
Exam tips: Model card vs. AI system inventory is a favorite swap: short summary of one model vs. the full internal repository (incident plans, data dictionaries, source links, contacts). · When a question asks how to forecast or limit secondary or unintended uses at release, the model card (intended and out-of-scope uses) is the classic answer. · Conformity happens BEFORE placing on the market. Any option that ships first and assesses later is wrong. · Unplanned substantial modification = new conformity assessment. Pre-determined, documented changes are the exception.
- ✓Monitor performance, inputs, outputs, fairness, security, human signals and operations.
- ✓Every metric needs a threshold and an owner.
- ✓Monitoring is a loop: collect, measure, alert, diagnose, act, validate.
- ✓Providers monitor the whole fleet (Art. 72); deployers monitor their own use (Art. 26).
- Continuous monitoring:
- Ongoing tracking of a deployed AI system's performance, inputs, outputs, fairness, security and human feedback.
- Post-market monitoring (EU AI Act Art. 72):
- The provider's duty to systematically collect and analyze real-world performance data over a high-risk system's lifetime, based on a documented plan.
- Ground truth:
- The verified correct outcome that predictions are compared against to measure real accuracy.
- Alert threshold:
- A pre-set value that, when crossed, signals a problem and triggers action by a named owner.
- Override rate:
- How often human reviewers reject or change the system's output; a rising rate is an early warning sign.
Exam tips: Monitoring never 'completes.' Options that scale it back after a stable year, a clean audit or 'perfect accuracy' are traps. · Provider monitoring is fleet-wide and feeds corrective action; deployer monitoring covers its own use in context. Check whose perspective the question takes. · Input monitoring alone misses concept drift. Prefer answers that also track outcomes against ground truth. · Third-party and pre-trained components you build on are monitored too, including for bad actors (NIST MANAGE 3.1 and 3.2).
- ✓A model is a snapshot; drift is the world moving away from it.
- ✓Data drift = inputs change; concept drift = input-outcome relationship changes; model drift = performance decays.
- ✓Maintain on a schedule AND on triggers.
- ✓Every update follows change management: diagnose, retrain, re-validate, check conformity, document, stage, monitor.
- Data drift:
- A change in the distribution of the inputs a model receives compared with its training data.
- Concept drift:
- A change in the relationship between inputs and the correct output, so the same inputs now call for a different answer.
- Model drift (model decay):
- The decline in a deployed model's performance over time, usually caused by data or concept drift.
- Retraining trigger:
- An automated threshold or alarm that starts the retraining process when monitored performance crosses a defined line.
- Change management:
- The controlled process for updating a system: test, version, document, approve and release with rollback ready.
- Champion–challenger:
- Testing a candidate model against the production model on live cases and promoting it only if it clearly performs better.
Exam tips: Inputs changed = data drift. Inputs look the same but outcomes behave differently = concept drift. Model drift (decay) is the performance symptom. · An alert justifies investigation, not automatic emergency retraining. Diagnose, then retrain through change management. · Don't confuse overfitting (a training problem) with drift (a post-deployment problem). · Retraining on a narrow, recent window can create temporal bias; the exam rewards representative data.
- ✓Four assessments, four questions: do we comply? can it be broken? what could go wrong? do the controls hold?
- ✓Threat model first; it scopes red teaming and security testing. Audits check the whole program against criteria.
- ✓High-stakes markers call for an independent or third-party audit.
- ✓Run them on a calendar and on triggers, scaled to risk.
- ✓Threat hunting, bug bashes and bug bounties supplement, never replace, internal testing and monitoring.
- Audit:
- An independent or internal check of a system or program against defined criteria such as laws, standards, policies or the provider's own claims.
- Red teaming:
- An active, creative simulation of adversarial attacks to find ways a system can be made to fail or misbehave.
- Threat modeling:
- Structured analysis, on paper, of a system's assets, entry points, likely attackers and techniques, used to prioritize defenses and scope testing.
- Security (penetration) testing:
- Structured testing of technical controls, such as APIs, infrastructure and access, to find exploitable vulnerabilities.
- Threat hunting:
- Proactively searching a live system for signs that someone is already abusing or compromising it, rather than waiting for alerts.
- Bug bash vs. bug bounty:
- A bug bash is a focused internal session where builders, testers and sometimes users hunt functional bugs together; a bug bounty pays outside researchers to report flaws responsibly.
Exam tips: Match the verb: audit CHECKS against criteria, red team ATTACKS, threat model MAPS on paper, security testing PROBES technical controls, threat hunting HUNTS for live abuse. · Threat modeling comes before red teaming and scopes it. Thinking about attackers is not yet attacking. · No assessment replaces another, or replaces continuous monitoring. 'X lets us retire Y' is a trap. · Independence is a hallmark of a credible audit. Consequential decisions, sensitive data, high-risk domains, scale, bias potential, critical infrastructure and public discourse are markers for an independent or third-party audit.
- ✓Main attacks: prompt injection, jailbreaking, data poisoning, evasion, model extraction, membership inference.
- ✓The OWASP Top 10 for LLMs ranks prompt injection first; MITRE ATLAS catalogs real attacker techniques.
- ✓Red teaming: scope, diverse team, attack, record, fix and retest, feed back.
- ✓Findings must flow into fixes, documentation and monitoring.
- Prompt injection:
- Crafted input that overrides a model's instructions, either typed directly or hidden in content the model reads (indirect).
- Jailbreak:
- A technique that coaxes a model into bypassing its safety rules or usage restrictions.
- Data poisoning:
- Corrupting training or fine-tuning data so the model learns harmful or attacker-chosen behavior.
- Model extraction:
- Reconstructing a working copy of a model by systematically querying it and learning from its outputs.
- Membership inference:
- Probing a model's outputs to determine whether a specific person's data was in its training set.
- Excessive agency:
- Giving an AI agent more permissions, tools or autonomy than its task needs, so a manipulation can do more damage.
Exam tips: Check the attacker's access: evasion, extraction and membership inference need only query access; poisoning needs a route into the training data. · Indirect prompt injection (instructions hidden in content the model reads) is a favorite GenAI scenario. · OWASP, MITRE ATLAS and NIST AI 600-1 are voluntary resources. Don't pick an option calling them legally binding. · A confirmed, unfixed high-severity finding means fix and retest before launch. Publishing exploit details is never the 'transparency' answer.
- ✓Risk = might happen; issue = a problem now; incident = harm or near harm. Log all three.
- ✓Prepare, identify, contain, eradicate, recover, learn, and notify as soon as a duty triggers.
- ✓EU high-risk providers report serious incidents fast, then investigate and take corrective action.
- ✓Preserve evidence and keep records accurate and complete.
- ✓Mine AI incident databases to learn from other people's failures.
- AI incident:
- An event in which an AI system caused, or nearly caused, harm to people, property, rights or the environment.
- Near miss:
- An incident where harm was narrowly avoided; it is still documented and investigated.
- Risk register:
- The living record of identified risks with their likelihood, severity, owner and mitigations.
- Serious incident (EU AI Act):
- An incident or malfunction leading to death or serious harm to health, serious and irreversible disruption of critical infrastructure, breach of fundamental-rights obligations, or serious harm to property or the environment.
- Containment:
- The response step that stops harm spreading, for example by disabling a feature or rolling back, while preserving evidence.
- AI incident database:
- A public collection of real-world AI harms and near misses (e.g., the AI Incident Database, OECD AIM) used for learning, not regulatory reporting.
Exam tips: Sequence: identify → contain → eradicate → recover → learn. Right after identification, the answer is containment, not root-cause surgery. · Speed must not destroy evidence: preserve logs and the model version, and under Art. 73 don't alter the system before informing authorities. · Art. 73 deadlines are outer limits (15 days; 10 for a death; 2 for widespread infringement or critical infrastructure). 'Immediately' is the rule. · Public incident databases are for learning. They are never the regulatory reporting channel.
- ✓Usual suspects: brittleness, lack of robustness, poor-quality data, insufficient testing, drift, adversarial attacks, hallucination, misaligned objectives.
- ✓Older outlines add catastrophic forgetting, uncertainty and false positives.
- ✓A label ('it hallucinated') is not a root cause; keep asking why.
- ✓Root-cause analysis is cross-functional because each team sees different causes.
- ✓Use structured tools (5 Whys, fishbone) in a blameless culture.
- Brittleness:
- A system's tendency to fail, often abruptly, when conditions differ from those it was trained on.
- Robustness:
- A system's ability to maintain performance across varied, noisy, unexpected or manipulated conditions.
- Hallucination (confabulation):
- A generative model producing fluent, confident output that is false or unsupported.
- Root cause analysis:
- A structured investigation that traces an incident past its symptoms to the underlying cause that must be fixed.
- Fishbone (Ishikawa) diagram:
- A root-cause tool that groups candidate causes into categories such as data, model, process, people and environment.
- Catastrophic forgetting:
- When retraining or fine-tuning on new data makes a model abruptly lose capabilities it had before.
Exam tips: 'Patch and close' and 'blame the vendor first' are traps. The IAPP answer documents the incident and runs cross-functional root-cause analysis before choosing a corrective action. · A known, foreseeable condition that was never tested points to insufficient testing, even if the failure looks like brittleness. · A suspiciously fast single-team diagnosis that matches that team's specialty is a cue for cross-functional review. · Hallucination applies to generative models; don't pick it for a classifier or scoring model. Old skills vanishing after a fine-tune or retrain is catastrophic forgetting, not drift.
- ✓Technical documentation proves compliance (regulators); instructions for use enable safe operation (deployers).
- ✓The post-market monitoring plan sits inside the technical documentation.
- ✓Transparency reports and model or system cards tell the public, and some laws now require them.
- ✓Disclosures are living documents with a boundary: enable safe use, don't arm attackers.
- Technical documentation (EU AI Act Art. 11):
- The detailed, up-to-date file showing how a high-risk system meets the Act's requirements, kept for authorities and set out in Annex IV.
- Instructions for use:
- Provider-written guidance that lets deployers operate a high-risk system correctly, including its purpose, accuracy, limits, risks and oversight needs (Art. 13).
- Post-market monitoring plan:
- The documented plan for how a provider will collect and analyze real-world performance data; part of the technical documentation (Art. 72).
- Transparency report:
- A public summary of a system's capabilities, limitations, testing and risk management, voluntary or required by some laws.
- EU database for high-risk AI:
- The public EU register where providers record high-risk systems, including electronic instructions for use for most of them (Arts. 49 and 71).
Exam tips: Match the document to the audience: technical documentation for regulators, instructions for use for deployers, transparency reports for the public. · If a deployer misuses a system because the instructions hid its limits, the provider's disclosure failed. Instructions for use are how providers enable correct deployment. · Transparency isn't 'publish everything.' Options that release exploit details, personal data or source code are traps. · The FRIA is a deployer duty. Don't pick it when the question asks what the provider owes.
- ✓Separate development, test or staging, and production; promote only through the pipeline.
- ✓Segregate duties so no one person can build, approve and deploy alone.
- ✓A model registry ties each version to its data, results, owner and status: traceable, repeatable, reversible.
- ✓Secure the pipeline: signed artifacts, restricted triggers, dependency scans, rogue-deployment alerts, tamper-evident logs.
- Environment separation:
- Keeping development, test or staging, and production apart, with models promoted only through an approved pipeline.
- Segregation of duties:
- A control ensuring no single person can build, approve and deploy a change alone.
- Model registry:
- The single record of every model version, with its hash, dataset versions, evaluation results, owner, approvals and status.
- Rogue deployment:
- A model running in production that didn't come through the approved pipeline or isn't in the registry.
- Artifact signing:
- Cryptographically signing model files so the pipeline can verify they haven't been altered before deployment.
Exam tips: Development, test or staging, and production stay separate; models move only through an approved pipeline. 'Copy it from the laptop' and 'give developers production admin' are traps. · Segregation of duties: nobody builds, approves and deploys alone. · Traceability question (which model and data made this decision?) → a model registry linking versions to datasets, kept after retirement. · An unregistered model in production is an incident: contain (roll back), preserve evidence, investigate. Never 'register it so the records match'.
Independent study aid, not affiliated with the IAPP. Written with AI from primary sources; laws change, so check the sources. Last updated 7 Nov 2026.
AIGP Quest