Retrieval-augmented generation (RAG)

What it means

An AI model learns from a huge amount of text, and then its learning stops. After that, it knows nothing about your company's policies, last week's contract, or this morning's price list. Retrieval-augmented generation, or RAG, is the usual fix. When someone asks a question, the system first searches a set of documents the organization chooses, then gives what it finds to the model to use in its answer. The US National Institute of Standards and Technology (NIST) describes it as a model "paired with a separate information retrieval system."¹

An everyday example: a new employee asked about the refund policy doesn't guess. They open the handbook, find the right page, and answer from it. IBM Research makes the same point with a picture from school: "It's the difference between an open-book and a closed-book exam."²

Where the word comes from

The term was coined in a 2020 research paper, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," first posted on May 22, 2020, by Patrick Lewis and eleven coauthors.³ Lewis's coauthors came "from the former Facebook AI Research (now Meta AI), University College London and New York University."⁴ The paper described "models which combine pre-trained parametric and non-parametric memory for language generation." In plain words, that means what the model learned in training, plus a library it can search. In the paper, that library was Wikipedia.³ The authors reported "more specific, diverse and factual language" than a model working from memory alone.³

Lewis has said he wishes the name were better: "We definitely would have put more thought into the name had we known our work would become so widespread." When it came time to write the paper, he said, "no one had a better idea."⁴ Since then the word has grown. It started as one specific research design. Now it covers almost any product that searches company documents before an AI writes an answer.

How others define it

• NIST: "A type of GenAI system in which a model is paired with a separate information retrieval system (or "knowledge base")." NIST adds: "RAG systems allow the internal knowledge of a GenAI model to be modified without the need for retraining."¹
• IBM Research: "RAG is an AI framework for retrieving facts from an external knowledge base to ground large language models (LLMs) on the most accurate, up-to-date information and to give users insight into LLMs' generative process."²
• Amazon Web Services: "Retrieval-Augmented Generation (RAG) is the process of optimizing the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response."⁵
• NVIDIA: "Retrieval-augmented generation is a technique for enhancing the accuracy and reliability of generative AI models."⁴

Where they agree and differ

All four agree on the core idea: look something up first, then answer. They differ on what they stress. NIST is neutral and describes how the system is built. Its definition comes from a NIST publication on attacks against AI systems, a reminder that the library itself can be tampered with.¹ IBM, AWS, and NVIDIA sell these products, and their definitions talk up the benefits: "most accurate, up-to-date," "authoritative," "accuracy and reliability."²⁵⁴

The strongest view against those benefit claims comes from Stanford researchers. They tested legal research tools that had been promoted as "eliminating" or "avoid[ing]" made-up answers.⁶ They found the tools "each hallucinate between 17% and 33% of the time" and concluded that "the providers' claims are overstated."⁶ (To hallucinate is to state something false with confidence; see [hallucination](/definitions/hallucination).) The fair reading: RAG can reduce errors, but it does not remove them. An answer is only as good as the documents found and how well the model reads them. We are confident in this reading, because the original paper, the vendors, and the critics all agree that grounding helps and none of them shows it is perfect.

In the news

Legal research tools, tested. In May 2024, the Stanford team tested AI research tools from LexisNexis and Thomson Reuters that rely on RAG.⁶ The result, quoted above, became a widely cited warning to lawyers and anyone else buying "grounded" AI. The tools tested were 2024 versions, and such products change over time.⁶

Agents need better retrieval. In January 2026, Fortune reported that Databricks had released a new retrieval design for AI agents. The article noted that "Since 2023, the standard way to do this has been with some kind of RAG, or retrieval augmented generation, process," and that "RAG is not a panacea."⁷ The accuracy gains Databricks reported came from the company's own tests, so treat them as claims, not independent findings.⁷

What it means for your organization

If a vendor tells you its AI is "grounded" in your documents, it almost certainly means RAG. That is good news, because answers can come from your current policies, and NIST notes the knowledge can be updated without retraining the model.¹ But it moves the hard question somewhere else. Which documents can the AI search? Who keeps them current? Who checks the answers? An [AI agent](/definitions/ai-agent) that searches stale or wrong files will be confidently wrong, and the Stanford results show that even good sources don't guarantee good answers.⁶ Treat the document library as something people own and maintain, and keep a person checking answers that matter (see [human in the loop](/definitions/human-in-the-loop)).

One question to ask your team: Which documents can our AI tools search today, who is responsible for keeping them accurate, and how often do we check that the answers actually match them?