Saltar al contenido principal
    Skip to main content

    RAG (retrieval-augmented generation)

    RAG (retrieval-augmented generation) is a technique that, before a language model answers, searches your documents for the passages that matter and passes them to the model along with the question. That way the model answers from your information, not only from what it learned in training.

    In practice

    According to Anthropic's glossary, in RAG the information is retrieved at the moment of each query and reaches the language model inside its context window, together with the original question. It is most useful when you need up-to-date information, knowledge specific to your business, or citations showing where each fact comes from. The term appears in the title of a 2020 research paper.

    A typical set-up has three steps:

    1. Chunking. Documents are split into pieces, usually no more than a few hundred tokens, according to Anthropic. OpenAI's Retrieval API indexes each file in 800-token chunks, each overlapping the next by 400.
    2. Indexing. An embedding model turns each chunk into a vector that encodes its meaning. Anthropic combines this with BM25, a word-matching search that does not miss exact codes or references.
    3. Retrieving and answering. For each question, the closest chunks are found and sent to the model with the query.

    Uses in a small business: an assistant that answers owners using the minutes and contracts of their own building, a chatbot that replies from your terms and FAQs, or an internal search tool that finds a contract clause. If the documents are scans, you need OCR first. There are examples in AI for property managers in Spain and in how much an AI chatbot costs.

    You do not always need it. In 2024 Anthropic explained that if the knowledge base is under 200,000 tokens, about 500 pages, you can put all of it in the request and skip RAG, using prompt caching to make it faster and cheaper.

    Limits

    • The answer depends on what is retrieved. Anthropic warns that RAG is only as effective as the quality and relevance of the knowledge base. Old or contradictory documents give old or contradictory answers.
    • It does not remove errors. Ask the model to cite the document behind each fact and to say "I can't find it" when it isn't there. Anthropic recommends both techniques and warns that they reduce made-up answers without eliminating them.
    • Documents can carry orders. A retrieved text can contain prompt injection, and OWASP points out that RAG does not fully prevent it.
    • One client, one space. The Spanish Data Protection Agency (AEPD) warns that using the same agent for different processing activities without separating their data can end up disclosing one person's data to others.

    Sources

    Articles that use this term