Ploba logo

Find the right AI tool for the job: apps, agents, APIs, MCP servers and datasets, explained plainly.

Get the AI Digest

Discover

  • Explore
  • Top Charts
  • Categories
  • Agents
  • MCP servers
  • Developer tools

Learn & community

  • Learn AI
  • AI Glossary
  • Blog
  • Hackathons
  • People
  • AI Digest

For makers

  • List your tool
  • Docs
  • Claim a listing
  • Advertise
  • Write for us

Company

  • About
  • Careers
  • Changelog
  • Trust
  • Security
  • Contact

© 2026 Ploba. All rights reserved.

  • Privacy
  • Terms
  • Cookies
  • Advertising
  • All policies
llms.txt
All articles
AI Concepts

What Is RAG? Retrieval-Augmented Generation Explained

Learn how Retrieval-Augmented Generation (RAG) connects AI models to your own documents, so answers are current, grounded and can cite where they came from.

Stephen Gbolagade

Sep 24, 2026 · 5 min read

Summarize with
ChatGPTClaudePerplexity

Mindmap of information
Image: Google Deepmind

If you ask a standard AI model to summarise your company's new HR policy, it will fail. It might invent a plausible-sounding policy, or apologise and say it doesn't know.

That's because a model only knows what was in its training data, up to a cut-off date. It has never seen the documents in your Google Drive, your help centre or your internal wiki.

RAG (retrieval-augmented generation) fixes that. Before the model answers, the system looks up the most relevant passages in your own documents and hands them to the model along with the question. The model then answers from that material instead of from memory.

In simple terms

RAG turns a closed-book exam into an open-book one. The AI still writes the answer, but it gets to look up the right pages first, and it can tell you which pages it used.

How RAG works

RAG has two halves: preparing your documents once, and answering each question.

Preparing the documents (once, then whenever they change)

  1. Collect: gather the sources the AI may use: PDFs, help articles, Notion pages, support tickets, product specs.
  2. Chunk: split each document into short passages, typically a few paragraphs each. Small chunks make it possible to pull out just the part that answers a question.
  3. Embed: turn each chunk into an embedding, a list of numbers that captures what the text means.
  4. Store: save the chunks and their embeddings in a vector database, along with details like the source, date and who's allowed to see it.

Answering a question (every time)

  1. Embed the question the same way the chunks were embedded.
  2. Retrieve: find the chunks whose meaning is closest to the question. This is semantic search: it matches meaning, not just exact words, so "Can I get my money back?" finds a passage titled "Refund policy". Closeness is usually measured with cosine similarity.
  3. Augment: put the best few chunks into the prompt, together with an instruction such as "Answer only from these sources and cite them."
  4. Generate: the model writes the answer, ideally pointing to the chunks it used.

A worked example

Say a customer asks your support assistant: "I bought the annual plan three weeks ago. Can I still get a refund?"

  • Without RAG, the model guesses from general knowledge about software refunds. It might say 14 days, or 30, or "contact support".
  • With RAG, retrieval finds two chunks: your refund policy ("Annual plans can be refunded in full within 30 days of purchase") and a note on how refunds are processed. The model answers: "Yes. Annual plans can be refunded within 30 days, so you're still eligible. Refunds go back to the original payment method within 5–10 business days." It links to the policy page it used.

If you change the policy tomorrow, you update one document. The next answer uses the new wording, with no retraining.

Why teams use RAG

Answers grounded in your own information

The model answers from documents you control, which cuts down hallucinations (confident, made-up answers). It doesn't remove them entirely: a model can still misread a passage or fill gaps around it, so important answers still need checking.

Always up to date

Training or fine-tuning a model is slow and costly, and the knowledge is frozen when it finishes. With RAG, updating the knowledge means updating a document.

Sources you can check

Because the system knows which chunks it retrieved, it can show citations. A reader can click through and confirm the answer, which builds trust. (Read more in Can you trust AI-generated citations?)

Respecting permissions

A well-built RAG system filters by access before retrieving, so a new employee's question never pulls in documents only finance is allowed to see.

RAG vs. Fine-Tuning

Wondering whether to use RAG or train your own model? Read our comparison of RAG, fine-tuning and prompt engineering.

Learn more

Where RAG goes wrong

Most problems with RAG come from retrieval, not from the model.

  • The right passage isn't found. If chunks are too long, too short or split in the wrong place, the answer may be spread across chunks that never get retrieved together.
  • The wrong passage is found. An outdated policy that's still in the index can outrank the current one. Old and duplicate documents need cleaning up.
  • Too much is retrieved. Stuffing many chunks into the prompt uses up the context window and costs more tokens, and the model may focus on the wrong one.
  • Some questions aren't search questions. "How many refunds did we issue last month?" needs a database query, not a paragraph. "Summarise every contract" needs all the documents, not the top three.
  • Retrieved text can carry instructions. A document that says "ignore previous instructions" is a prompt injection risk. Treat retrieved text as information, never as commands.

Teams improve results by tuning chunk size, combining semantic search with keyword search, re-ranking the results before they reach the model, and testing with real questions.

When not to use RAG

  • The model already knows the answer. General knowledge ("What is a CSV file?") doesn't need retrieval.
  • You need a new skill or style, not new facts. Teaching a model a format, tone or task is a job for prompting or fine-tuning.
  • The data is structured. Numbers in a database are better answered with a query (often through tool calling) than by searching text.
  • There's very little content. If everything fits comfortably in the prompt, simply include it.

Agentic RAG

Early RAG was a single pass: search once, then answer. Many systems now let an AI agent drive retrieval. The agent decides what it needs, searches, reads the results, rephrases and searches again if the first attempt missed, and can combine sources, such as a vector database, a live SQL database and a knowledge graph of how things relate.

This handles harder questions but is slower and costs more, so it's worth using where a single search regularly fails. It's also how agents keep a long-term memory; see How AI agent memory works.

Frequently asked questions

What does RAG stand for?

Retrieval-augmented generation: the system retrieves relevant text, augments the prompt with it, then the model generates the answer.

Does RAG stop AI hallucinations?

It reduces them by giving the model the right information, but it doesn't eliminate them. Good RAG systems also tell the model to say "I don't know" when the sources don't contain the answer, and show citations so people can check.

Do I need a vector database for RAG?

Most RAG systems use one, because it makes semantic search fast. Small projects can start with a simple keyword search, or a database that supports both.

Is RAG better than fine-tuning?

They solve different problems. RAG adds knowledge that changes; fine-tuning changes how a model behaves. Many products use both.

Does RAG keep my data private?

RAG itself doesn't send your data anywhere new beyond the model you call, but the retrieved passages are part of each request. Check how your model provider handles data, and filter by permissions before retrieval.

Summary

  • RAG looks up relevant passages in your own documents and gives them to the model before it answers.
  • It relies on chunking, embeddings, a vector database and semantic search.
  • It makes answers current and checkable and reduces made-up facts, but doesn't remove them.
  • Most failures are retrieval failures: wrong, missing, outdated or too many chunks.
  • Use it for knowledge that changes; use prompting or fine-tuning for behaviour and style.

About the author

Stephen Gbolagade

Software Engineer and Technical Writer at Ploba

Writes Ploba's guides to AI tools, agents and protocols, explaining how they work in plain language.

More from Stephen

Keep learning

Plain-language explanations, technical guides and practical comparisons.

More articlesAI terms glossary

On this page

  • How RAG works
  • Preparing the documents (once, then whenever they change)
  • Answering a question (every time)
  • A worked example
  • Why teams use RAG
  • Answers grounded in your own information
  • Always up to date
  • Sources you can check
  • Respecting permissions
  • Where RAG goes wrong
  • When not to use RAG
  • Agentic RAG
  • Frequently asked questions
  • What does RAG stand for?
  • Does RAG stop AI hallucinations?
  • Do I need a vector database for RAG?
  • Is RAG better than fine-tuning?
  • Does RAG keep my data private?
  • Summary

More in AI Concepts

See all
  • Scrabble letter tiles spelling a word on a marble surfaceAI Concepts · Nov 13, 2026What Are AI Tokens, and Why Do They Decide What You Pay?Tokens are the unit every AI model reads and bills by. This guide explains what they are, how counting works, and what to check when comparing API costs.
  • A building under construction wrapped in scaffolding against a blue skyAI Concepts · Nov 3, 2026What Is Harness Engineering? The System Around an AI AgentHarness engineering is designing everything around an AI model so an agent works reliably: instructions, tools, memory, checks and permissions.
  • Illustration of a person typing a prompt into an AI interfaceAI Concepts · Sep 3, 2026What Is Prompt Engineering? A Practical Beginner’s GuideLearn the basics of prompt engineering. Discover practical tips for talking to AI models like ChatGPT and Claude to get better, more accurate results.