Retrieval-augmented generation

In plain English

Think of a capable new hire who knows a lot about the world and nothing about your company. RAG hands them the right pages from your filing cabinet just before they answer each question.

It works in four steps:

1. Documents are split into chunks and converted into numeric representations (embeddings).
2. These are stored in a searchable index, often a vector database.
3. A user's question is matched against the index, and the best passages are retrieved.
4. The passages are added to the prompt, and the model answers, ideally citing them.

The model's weights stay the same. RAG changes what the model can see, not what it is.

Example

Pemberton City Council, a fictional local authority, builds an HR assistant over 2,000 policy documents.

• Permission-aware retrieval. Staff can only draw on documents they could already open. Payroll and case files are excluded.
• Content ownership. Every document has an owner and a review date. Withdrawn policies leave the index the same day.
• Untrusted content. Uploaded files are scanned, and retrieved text can never trigger actions.
• Citations. Each answer links to the paragraph it relied on, so staff can check.

Six months in, the assistant quotes a superseded leave policy. The root cause is a stale index, so the council fixes its content process rather than the model.

Why it matters

Origin. The term comes from a 2020 paper by Lewis and colleagues. It combined a model's built-in knowledge with a retrievable document index.

NIST. The Generative AI Profile names RAG as a factor when identifying a system's intended purpose (MP-1.1-001). It also suggests re-assessing model risks after implementing RAG or fine-tuning (MG-3.1-003).

Security. OWASP's LLM08:2025 covers weaknesses in how embeddings are generated, stored and retrieved. NIST AI 100-2 explains how attackers plant instructions in resources a RAG system retrieves (prompt injection). OWASP notes that RAG does not fully mitigate prompt injection.

Data protection. If the index holds personal data, the usual rules on purpose, minimization and access apply. Deleting a document from an index is far easier than making a fine-tuned model unlearn it.

How the AIGP exam tests it

RAG sits in the adaptation ladder of lesson IV.A.4: as is, prompts, RAG, fine-tuning, agents.

Common traps:

• "RAG retrains the model." It does not touch the weights.
• RAG or fine-tuning? Changing content, citations and per-user permissions point to RAG. A stable house style at huge volume points to fine-tuning. See RAG vs fine-tuning.
• "A withdrawn document must stop influencing answers today." Delete it from the index. That is RAG's advantage.
• "RAG ends hallucination." It reduces AI hallucination but turns the corpus into the risk.

The tell: govern the document store like production data, with owners, quality checks, permissions and freshness.