IndicRAG: The Jump from Standard RAG to Agentic RAG

← Back to Blog

The Ceiling of Standard RAG

The first version of IndicRAG was a standard RAG pipeline: embed the query, retrieve chunks, rerank, generate an answer, done. One pass, no second-guessing. Through v1.5.0 that pipeline grew a lot of real capability — hybrid dense + BM25 retrieval, cross-encoder reranking, a faithfulness verifier — but the shape stayed the same: retrieve once, answer once, hope the top-k was enough.

That shape has a hard ceiling. A single retrieval pass can't decide mid-answer that it needs a different source. It can't check its own output for claims the retrieved chunks don't actually support. And it can't reach outside the indexed corpus when the honest answer is "go search arXiv," not "here's the closest match from what I've already indexed."

v2.0.0 — What Actually Changed

The release notes for v2.0.0 don't undersell it: this upgraded IndicRAG "from a standard retrieval pipeline to a full Agentic Scientific RAG system with multi-tool orchestration, reflexion-based quality control, LLM failover, and defense-in-depth sandboxing." Four pieces did the actual work.

1. A LangGraph State Machine Instead of a Function Call

The new /agent/query endpoint runs a graph with six nodes: query_planner → tool_selector → tool_executor → answer_generator → reflexion_evaluator → finalizer. Unlike a straight-line pipeline, this graph has conditional edges — the reflexion evaluator can route execution back into tool_executor or answer_generator instead of finishing.

2. Six Tools, Not One

Standard RAG had exactly one move: search the indexed corpus. The agent has six:

• indicrag_retrieval — the original hybrid search, now with query expansion
• arxiv_search — live arXiv lookup
• open_access_search — Semantic Scholar with automatic OpenAlex fallback
• web_search — Tavily web search for non-academic context
• calculate — numexpr-based math evaluation
• execute_python — sandboxed Python execution

Selected tools run concurrently through a ThreadPoolExecutor, capped at 4 concurrent tools, which the release notes clock as saving 3-10 seconds per query compared to running them sequentially.

3. A Reflexion Loop That Can Say "Not Good Enough"

After the answer generator produces a draft, the reflexion_evaluator scores it on two axes: faithfulness (an NLI model checking whether claims are actually entailed by the retrieved evidence) and completeness (judged by Gemini). Below threshold, the graph can trigger auto-regeneration. A stuck-loop detector auto-accepts once completeness stops improving by more than 0.05, and falls back to a safe response if it's stuck with low faithfulness — so the loop can't spin forever chasing a score it'll never hit.

4. Failover Before It Was Cool

v2.0.0 also shipped generate_with_failover(): round-robin through all configured API keys on the primary model, then drop to a fallback model on repeated 503/429s. A circuit breaker skips the primary model for 60 seconds after all keys fail, which the release notes say avoids about 15 seconds of wasted retries per call during an outage. This is the same idea that gets generalized to multi-provider routing two releases later in v2.3.0.

Security got a matching upgrade: execute_python moved from a string denylist to an AST-based sandbox — parse the code, walk the tree, whitelist imports, block dunder names/attributes, block string-formatting escape vectors — running under a restricted __builtins__ and a restricted __import__. The frontend started sanitizing all rendered HTML through DOMPurify before innerHTML assignment, closing an XSS path that a naive markdown renderer would otherwise leave open.

Hardening the Leap: v2.0.1 through v2.3.0

Rewriting the core doesn't ship a stable system on day one. The next four releases are almost entirely about finding what the rewrite broke and closing the gaps — which is exactly what you'd expect after replacing a straight-line pipeline with a stateful agent loop.

Version	Focus	What landed
v2.0.1	Reflexion bug fixes	Fixed an unescaped brace in the completeness prompt that silently threw on every .format() call, forcing a fake 0.50 completeness score and triggering stuck-loop on every query. Fixed the faithfulness verifier reading the wrong logit index (neutral instead of entailment), which had made faith=0.00 on every iteration — the faithfulness gate could never pass.
v2.1	13 production bugs	BM25 tokenizer was stripping Indic combining marks, degrading search across all 11 Indic languages. Strategy B was leaking the untranslated query into the prompt. SSE streams could hang or hit phantom done events. ChromaDB calls got timeout-safe shutdown. Safety settings defaulted to BLOCK_MEDIUM_AND_ABOVE unless explicitly overridden.
v2.2.0	Retrieval depth + production hardening	Section-aware per-paper retrieval scoping, ColBERT late-interaction reranking, HyDE query expansion. Reflexion gained a wall-clock budget on top of the iteration cap. Constant-time API-key auth, per-key rate limiting, least-privilege CI, non-root Docker image.
v2.3.0	Multi-provider + multimodal (latest)	OpenRouter added alongside Gemini with per-(provider, model) circuit breaking and a guaranteed Gemini backstop. Figure/table indexing at ingest, surfaced as clickable thumbnails. Confidence scoring with explicit abstention. Cross-source contradiction detection via pairwise NLI. Topic watches and async literature-review reports.

The Bugs Are the Interesting Part

Two fixes from v2.0.1 are worth sitting with, because they're the kind of bug that doesn't crash anything — it just quietly makes the new feature worthless. The completeness prompt had a literal { brace that Python's .format() read as a field marker, threw a ValueError, and the exception was caught and swallowed — so the reflexion loop always saw a hardcoded fallback score instead of a real judgment. The faithfulness scorer had the right model but the wrong array index, so faith was always reported as zero. Both bugs meant the reflexion loop technically ran on every single query, and technically never worked, for the entire lifetime of that release. Nothing in the logs looked broken. The agent just never got better at judging its own answers.

That's the honest argument for shipping a reflexion loop with a hard iteration cap and a stuck-loop fallback in the first place: even when the self-critique step is silently broken, the system still returns an answer instead of hanging.

Multi-Provider LLM in v2.3.0

v2.0.0's failover only round-robinned within one Gemini account. v2.3.0 generalizes that into a real multi-provider layer: Gemini and OpenRouter (Claude, GPT, Llama, and others), selectable per request via a GET /models endpoint that returns a curated allowlist with tool-capability metadata, so the UI can grey out models that can't call tools.

• Per-(provider, model) circuit breaker — one model's rate limit doesn't take down every model behind the same provider
• Guaranteed Gemini backstop — any selected OpenRouter model still resolves when its free-tier endpoint 429s, instead of failing the request
• Provider-aware JSON retry — when a non-Gemini model returns malformed structured output, the retry routes through Gemini's stricter parser instead of failing outright

The response cache key also grew from prompt + model to prompt + model + provider, so the same model name routed through two different providers no longer collides in the cache — a bug that would otherwise have been invisible until someone compared two answers that should have differed and didn't.

Multimodal Grounding and Contradiction Detection

v2.3.0's other major addition is indexing figures and tables at ingest time, not just text — crops get extracted, captioned, and embedded alongside text chunks, and the API/UI can now cite a specific figure or table crop inline with an answer instead of just a text span. On the trust side, the answer generator runs pairwise NLI across retrieved sources to flag contradictions instead of silently picking one side, and a finalizer computes an explicit confidence score — low-confidence answers get an abstention prefix rather than a fabricated-sounding answer with no signal that it's shaky.

Two async subsystems round out v2.3.0: topic watches (/watch/*) that run on a daily/weekly/monthly cadence and persist digests of new papers, and literature review reports (/report/*) that decompose a topic into sections and generate a cited Markdown report with document-global citation numbering — one number per paper across the whole report, not a new number every section.

Retrieval Quality, Measured

Through all of this, the underlying hybrid retrieval — BGE-M3 dense vectors fused with BM25 via Reciprocal Rank Fusion, reranked by bge-reranker-v2-m3 — validates against manually labeled relevance judgments across 10+ Indian languages:

Key Takeaways

The jump from Standard RAG to Agentic RAG wasn't "wrap the existing retriever in an LLM loop." It was a full rewrite of the core around a stateful graph, plus a whole second track of work — failover, sandboxing, circuit breakers, rate limiting — that has nothing to do with retrieval quality and everything to do with the system surviving contact with a production workload. The three releases after v2.0.0 spent as much effort fixing what the rewrite broke as v2.0.0 spent building it. That ratio is normal for this kind of architectural jump, and pretending otherwise is how "agentic" systems ship broken reflexion loops that nobody notices for a full release cycle.

Interested in agentic RAG systems and production ML infrastructure? Let's connect!

← Back to Blog