Retrieval-augmented generation (RAG), explained for business teams
A language model knows what it was trained on, and nothing about your price list, your contracts or last week’s policy change. Retrieval-augmented generation, or RAG, closes that gap. Before answering, the system looks up the relevant passages in your own documents and gives them to the model to answer from.
By the FocalID team in Colombo, , 2 min read

How RAG works
A RAG system does its work in two stages.
- Preparation: your documents are split into passages, and each passage is indexed so it can be found by meaning, not only by exact words.
- Retrieval: when someone asks a question, the system finds the handful of passages most relevant to it.
- Generation: the model writes an answer using those passages, and can point to the documents it used.
Why not just train the model on our documents?
Training or fine-tuning a model on your documents is slower, costs more and has to be repeated when the documents change. With RAG, you update the document and the next answer uses the new version. You can also control access: the system retrieves only what the person asking is allowed to see.
Fine-tuning still has a place, for teaching a model a narrow task or a house style. The two are often used together.
Where it works well
- Staff questions on HR policy, procedures and product manuals.
- Customer support answers drawn from your own help content.
- Finding clauses and precedents in contracts and legal documents.
- Giving an AI agent the background it needs before it acts, such as the returns policy for a refund request.
What makes RAG fail
Most poor RAG systems fail at retrieval, not at the model. The usual causes are:
- Out-of-date or duplicate documents, so the system finds the old version.
- Scanned PDFs and tables that were never converted to proper text.
- Passages cut in the wrong places, so the answer is split across two pieces.
- No testing: nobody has checked a set of real questions against known correct answers.
Getting started
Pick one set of documents with a clear owner, clean it up, and collect fifty real questions with their correct answers. Build the system, test it against those questions, and ask it to show its sources on every answer so users can check them. Expand to more documents only when the first set answers reliably.
Common questions
Does RAG stop an AI from making things up?
It reduces the problem by giving the model real passages to answer from, and showing sources lets users check. It does not remove the need for testing.
Do our documents leave our servers?
They do not have to. The index and the model can both run on your own servers or private cloud where that matters.
How many documents do we need?
There is no minimum. A few well-kept documents give better answers than thousands of outdated ones.

