Retrieval-augmented generation, or RAG, is a way of building AI answers in two steps. First a search system retrieves documents that match the question. Then a large language model writes the answer using those documents as its source material, often with links to them.
Without retrieval, a model can only draw on what it absorbed in training. That knowledge stops at a cutoff date, can't include your private documents, and comes with no record of where a fact came from. Retrieval fixes those gaps by handing the model the relevant text at the moment it answers.
The name comes from a 2020 research paper by Patrick Lewis and colleagues, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks." It described a model with two kinds of memory: what the trained model itself knows, and an external index of Wikipedia passages it could search. The models it tested wrote more specific and more factual text than the same kind of model without retrieval.
How a RAG system answers a question
Most systems follow the same path, whether they answer from a company handbook or the whole web.
- Prepare the documents. Split them into passages (chunks) and store them in a searchable index. Many systems turn each passage into an embedding, a list of numbers that captures its meaning, and keep those in a vector database.
- Retrieve. When a question comes in, search the index for the passages closest to it. Good systems combine meaning-based vector search with keyword search and then re-rank the results.
- Augment the prompt. Put the best passages into the prompt together with the question and instructions, such as "answer only from these sources and cite them."
- Generate. The large language model writes the answer from that material.
Updating what the system knows means updating the index, not retraining the model. Add a new policy document today and the next answer can use it.
RAG or fine-tuning?
People often weigh RAG against fine-tuning, which trains a model further on your own examples. They solve different problems:
| You need the model to | Better fit |
|---|---|
| Use facts that change, such as prices, stock or news | RAG |
| Answer from private documents and show which one it used | RAG |
| Write in a fixed style or format, or follow a specialized task pattern | Fine-tuning |
| Do both | Fine-tune for behavior, retrieve for facts |
For many teams the cheaper first step is neither: clear instructions and examples in the prompt. See prompt engineering.
Where RAG goes wrong
RAG reduces made-up answers, but it doesn't end them. Most failures trace back to retrieval.
- The wrong passages come back. If the search finds irrelevant text, the answer is grounded in the wrong source: confident, cited and off-topic.
- The right passage is split badly. A chunk that cuts a table in half or separates a rule from its exception gives the model half the story.
- The index is stale. The model can only be as current as the last update of its documents.
- The model ignores or blends its sources. It can still mix retrieved facts with what it learned in training, so citations need checking.
Evaluating a RAG system means testing both halves: did it retrieve the right passages, and did the answer stay faithful to them?
Why RAG matters for SEO
AI search engines work on the same principle at web scale. Google calls the step "grounding": in its own words, providing content from the Google Search index to the model at prompt time to improve factuality and relevancy. Google For AI Overviews and AI Mode, Google describes a "query fan-out": the system may run several related searches across subtopics and identify supporting pages while the response is being generated.
For a site owner that has practical consequences:
- Your page can only be used if it can be retrieved. Google says a page must be indexed and eligible to show with a snippet to appear as a supporting link in AI Overviews or AI Mode. There are no extra technical requirements, no special file and no special schema markup.
- Passages matter, not just pages. Retrieval picks pieces of text. A section that states its answer plainly, under a heading that names the question, gives a retriever a cleaner match than one that builds up to the point.
- You keep some control. In Google Search,
nosnippet,data-nosnippet,max-snippetandnoindexlimit what can be shown from your pages. The separate Google-Extended token in robots.txt controls whether Google may use your content to train future Gemini models and to ground answers in Gemini Apps and Grounding with Google Search on Vertex AI. It doesn't affect Google Search.
Other AI assistants that search the web may rely on their own crawlers and indexes, with their own rules, so check the documentation of each one you care about. For how to measure whether these systems mention you, see AI visibility and generative AI.