Retrieval-Augmented Generation (RAG)

How RAG works

A language model's built-in knowledge is frozen at training time and can't include your private documents or today's facts. RAG fixes this by adding a retrieval step before generation: the system takes the user's question, searches a knowledge base — often a vector database of embedded documents — for the most relevant passages, and inserts those passages into the model's prompt as context. The model then answers using that retrieved material rather than guessing from memory.

The payoff is grounding. Because the answer is built from specific, retrieved sources, the model is less likely to hallucinate and can cite where each claim came from. This is why research agents like Elicit and GPT Researcher feel trustworthy — they retrieve real papers or web pages and synthesize from them, rather than producing unsourced prose.

Why it matters for agents

RAG is what lets an agent work over knowledge it was never trained on — a company's internal wiki, a product's documentation, a live help center, or a corpus of academic papers. Customer-support agents use it to answer from your help articles and policies; research agents use it to read sources on demand. Without retrieval, an agent is limited to general knowledge and prone to confident errors on anything specific.

RAG isn't a complete safeguard, though. The answer is only as good as what's retrieved — poor search returns poor context — and retrieved content is also a vector for prompt injection if it isn't treated as untrusted. Good RAG systems invest as much in the quality and security of retrieval as in the model itself.

What is retrieval-augmented generation (RAG)?

RAG is a technique where an AI model retrieves relevant documents from an external knowledge source and uses them as context to ground its answer, instead of relying only on training data. It lets agents answer from current, private, or specialized information and cite sources.

Why does RAG reduce hallucination?

Because the model builds its answer from specific passages retrieved from a real knowledge base rather than guessing from memory, it's less likely to invent facts and can point to where each claim came from. The grounding is only as good as the retrieval, though.

How do AI agents use RAG?

RAG lets an agent work over knowledge it wasn't trained on — a help center, internal docs, or academic papers. Support agents answer from your policies and research agents read sources on demand, which is what makes their answers specific and citable.