Agentic RAG

In traditional RAG, retrieval is a fixed, predetermined step. In Agentic RAG, the agent itself decides when, how many times, and how to search.

What Is Agentic RAG?

Agentic RAG combines RAG with agentic behavior: retrieval is no longer a fixed step in a linear pipeline, but is itself a tool available to the agent, which the agent can use zero, one, or several times (with different queries) based on need.

Difference from Plain RAG

Key Patterns

• Query Rewriting — rewriting the user's ambiguous question into a more precise search query before retrieval
• Multi-hop Retrieval — answering a complex question requires several sequential searches (the first search's result shapes the second search's input)
• Self-correction / Critique — the agent critiques the retrieved results; if they're not relevant or sufficient, it tries again with a different query

A Real-World Flow

Suppose the user asks: "What's the cheapest headphone that has noise cancellation and is in stock?"

1. The agent decomposes the question into two sub-questions: "noise-cancelling headphones" and "checking each one's stock"
2. The first search (RAG over the catalog) returns candidate results
3. The agent calls the check_inventory tool for each one (here, RAG and MCP work side by side)
4. If the initial results are all empty, the agent rewrites the query and searches again
5. The final answer, the cheapest in-stock option, is given to the user

When Do You Need It?

If your users' questions are usually simple and single-dimensional, traditional RAG is sufficient and cheaper. If questions are multi-step, ambiguous, or require combining several data sources (like the example above), Agentic RAG noticeably improves accuracy — at the cost of added latency and compute.

Guarding Against an Infinite Search Loop

Because in Agentic RAG the agent itself decides whether to search again, the main risk is that for questions with no answer at all in the knowledge base, the agent keeps rewriting the query and searching again, over and over, without ever reaching a result — exactly the same "infinite loop" risk raised in AI Agents. Common ways to guard against it:

• An attempt cap — limit re-searches to a fixed number (e.g. a maximum of 3) before returning an "insufficient information found" answer
• A quality threshold — if the relevance score of retrieved results is still below a threshold after N attempts, the agent should stop instead of continuing aimlessly
• Transparency with the user — instead of guessing, explicitly state that a definitive answer wasn't found in the available sources

FAQ