What does RAG stand for?

RAG stands for Retrieval-Augmented Generation. It describes a system that, before letting a language model write an answer, retrieves the passages relevant to the question from a document collection and provides them as the basis for the answer.

How does a RAG system work in practice?

In four stages. Documents are uploaded and their text extracted, via OCR if they are scans. The text is split into chunks and turned into numeric vectors stored in a vector database. When a question arrives it is turned into a vector too, and the closest chunks are retrieved. Those chunks then go to the language model, which writes the answer citing its sources.

What is the difference between RAG and fine-tuning?

Fine-tuning changes the model's behaviour by retraining it, and suits changing the style or format of answers. RAG does not touch the model: it changes the information placed in front of it at question time. To make an AI aware of company documents that change over time, RAG is the correct approach, because reindexing replaces retraining.

Does RAG eliminate hallucinations?

It reduces them substantially but does not eliminate them. The decisive advantage is different: because every answer arrives with the passages it was drawn from, a wrong statement can be checked in seconds instead of being indistinguishable from a correct one.

Does RAG need an internet connection?

Not if every component is local. Text extraction, embedding, the vector database and the language model can all run on one machine. The I3K RAG Enterprise Community engine fetches its components on first run and works without a network from then on.

How many documents can a RAG system handle?

That depends on hardware and on the vector database rather than on the language model: document count drives memory use and indexing time, while the cost of a single answer stays roughly constant because only a few passages ever reach the model. The serious way to answer is to measure on your own corpus and your own hardware.

Concepts

What RAG (Retrieval-Augmented Generation) is, and why it matters to a business

RAG is the technique that lets a language model answer questions about documents it never saw in training: it retrieves them first, then writes an answer citing what it found.

It is the most direct way to put AI to work on company material without retraining anything and without handing your archives to a vendor. This page explains how it works, how it differs from fine-tuning, and where it breaks.

The definition, in two sentences

RAG stands for Retrieval-Augmented Generation. Instead of asking a language model to answer from memory, the system first retrieves the relevant passages from a collection of documents and hands them to the model along with the question, asking it to answer only from those.

The consequence is that the model stops being the source of the answer and becomes its editor. The source is your documents, and every statement can be traced back to the exact passage it came from.

The shortest way to put it: without RAG you ask a model what it remembers; with RAG you ask it to read and report.

Why it is needed, when models already know so much

Because they know nothing about you. A language model was trained on public text up to some date: your contracts, your reports, your internal policies and last year's minutes are not part of it and never will be.

There are then three problems RAG solves that using a model directly does not. The first is freshness: documents change, and reindexing them takes minutes while retraining a model costs far more. The second is verifiability: if you cannot see where an answer came from, you cannot use it for a decision you will have to defend. The third is access control: because passages can be filtered before they reach the model, a person never receives an answer built on documents they could not have opened.

The four stages, in order

01
Ingest
Upload from the web UI or the REST API. PDF, DOCX, XLSX, HTML, TXT, Markdown and CSV. Scanned pages are detected automatically and passed through OCR in Italian and English.
Tika · OCR
02
Embed & store
Text is split into overlapping chunks and embedded with BAAI/bge-m3 (1024 dimensions, 100+ languages), running in-process. Vectors go into Qdrant with the metadata used for role-based filtering.
bge-m3 · Qdrant
03
Retrieve
The question is embedded the same way and answered from the closest passages. Relevance threshold and top-K are configurable, and role filtering is applied here — a query never reaches the model with passages the caller may not see.
I3K orchestrator
04
Generate
The passages go to the local language model, which streams an answer token by token with each source shown. The model runs on your hardware. Zero external calls.
EuLLM · Mistral 7B

See the technology

RAG or fine-tuning: not the same question

They are often presented as alternatives, but they do different jobs. Fine-tuning changes the model's behaviour: tone, format, the way it approaches a kind of task. RAG changes the information the model has in front of it at the moment it answers.

If the problem is “it does not know our documents”, the answer is RAG. Fine-tuning on a company corpus is an expensive and unreliable way to memorise facts.
If the problem is “it answers in the wrong way”, look at the prompt first and only then at fine-tuning.
If the documents change often, RAG is the only workable option: you reindex, you do not retrain.
If you need to cite the source, you need RAG: a model trained on a text cannot tell you which page a claim came from.

What answer quality actually depends on

Almost never the language model. When a RAG system answers badly, in the vast majority of cases it retrieved the wrong passages — and no model, however large, can write a good answer from the wrong material.

Where the game is won

How a document is split into chunks. A chunk cut in the wrong place breaks a clause in half and makes it unretrievable.
How much context each chunk carries. A paragraph pulled from page 340 with no idea which chapter it belongs to is ambiguous even to a human reader.
The embedding model — how text becomes comparable numbers. If it is multilingual, an Italian question can find an English passage.
The quality of text extraction. On a scanned PDF with no OCR there is nothing to retrieve, because there is no text.
How results are ordered before they reach the prompt: the first passages count far more than the last.

This is why a carefully built, self-hosted RAG system can beat a service built on a far more powerful model with mediocre retrieval. Model size is the factor people notice most and that matters least.

Where RAG breaks

Questions that require reading everything: “summarise the position taken across all 4,000 minutes” is not a retrieval question, because no three passages contain it.
Aggregate questions: counting, summing, grouping by date. Semantic retrieval finds similar text, it does not compute totals — that needs structured metadata alongside the vectors.
Contradictory documents: if the archive holds two versions of a procedure, the system may cite the superseded one. The fix is governing the archive, not the model.
Badly phrased questions: if the question shares neither terms nor a nearby concept with the document, retrieval has nothing to grip.

None of these is an implementation defect: they are the shape of the problem. Knowing them up front is the difference between a project that starts with the right expectations and one judged a failure at three months.

Why RAG is the shortest path to AI that never leaves the company

Because none of its stages needs an external service. Text extraction, embedding, the vector database and the model that writes the answer can all run on the same machine, and all of them are available as open-source software.

That is exactly what the I3K RAG Enterprise Community engine does: a single executable that starts the vector database and the inference engine itself, computes embeddings in its own process and, after the first run, needs no network at all.

Frequently asked questions about RAG

What does RAG stand for?: RAG stands for Retrieval-Augmented Generation. It describes a system that, before letting a language model write an answer, retrieves the passages relevant to the question from a document collection and provides them as the basis for the answer.
How does a RAG system work in practice?: In four stages. Documents are uploaded and their text extracted, via OCR if they are scans. The text is split into chunks and turned into numeric vectors stored in a vector database. When a question arrives it is turned into a vector too, and the closest chunks are retrieved. Those chunks then go to the language model, which writes the answer citing its sources.
What is the difference between RAG and fine-tuning?: Fine-tuning changes the model's behaviour by retraining it, and suits changing the style or format of answers. RAG does not touch the model: it changes the information placed in front of it at question time. To make an AI aware of company documents that change over time, RAG is the correct approach, because reindexing replaces retraining.
Does RAG eliminate hallucinations?: It reduces them substantially but does not eliminate them. The decisive advantage is different: because every answer arrives with the passages it was drawn from, a wrong statement can be checked in seconds instead of being indistinguishable from a correct one.
Does RAG need an internet connection?: Not if every component is local. Text extraction, embedding, the vector database and the language model can all run on one machine. The I3K RAG Enterprise Community engine fetches its components on first run and works without a network from then on.
How many documents can a RAG system handle?: That depends on hardware and on the vector database rather than on the language model: document count drives memory use and indexing time, while the cost of a single answer stays roughly constant because only a few passages ever reach the model. The serious way to answer is to measure on your own corpus and your own hardware.

Ready to run RAG on your own infrastructure?

Start with the open-source Community edition, or talk to us about Pro with structured extraction, SSO, audit log and SLA.

Star on GitHub Book a call