Skip to content

Research: Evals & Guardrails for Agentic RAG

TL;DR

  • Build an agent that is measurably accurate and safely bounded. This guide covers datasets, offline evals, online monitoring, and runtime guardrails in one connected system.
  • Follow the real engineering order: understand the failure modes → build a dataset → measure offline → add runtime guardrails → monitor live traffic → feed failures back into your dataset.
flowchart LR
    F["00 Foundations<br/>mental model + failure modes"] --> D["01 Datasets<br/>SME, synthetic, production"]
    D --> OFF["02 Offline evals<br/>end-to-end + component"]
    OFF --> G["04 Guardrails<br/>runtime enforcement"]
    G --> ON["03 Online evals<br/>+ observability"]
    ON -->|"traces, feedback, failures"| D
    DF["05 Decision framework<br/>what to do, in what order"] -.-> D
    DF -.-> OFF
    DF -.-> G
    DF -.-> ON
    T["06 Tooling"] -.-> OFF
    T -.-> G
    T -.-> ON

Start here (pick your goal)

If you are… Read this first Read this second
New to evals and guardrails The Evals & Guardrails Map Why Evals and Guardrails → Failure Modes
Building your first RAG or agent prototype Implementation Playbook Dataset Strategy → End-to-End RAG Evals
Choosing which metrics to run Eval Selection Decision Tree Metric Cheatsheet
Choosing which guardrails to add Guardrail Selection Decision Tree Guardrails Architecture
Getting ready for launch Implementation Playbook: Launch Readiness Threat Model & OWASP
Debugging a drop in live quality Monitoring, Alerting & Drift Production Data Flywheel
Preparing for a security or RBAC review Access Control & RBAC Tool & Action Guardrails
Benchmarking your team's maturity Maturity Model Implementation Playbook
Comparing open-source and vendor tools Eval & Observability Tools Guardrail Tools

The core mental model in one table

Every eval and guardrail in this repository is defined by six choices:

Choice Your options Where to learn more
1. When it runs • Offline (on datasets before deploy)
• Online (async on live traffic)
• Runtime guardrail (inline on live requests)		 The Map
2. Data it uses • SME golden dataset
• Synthetic dataset
• Production-sampled traces		 01 · Datasets
3. Scope • End-to-end (question → final answer)
• Component (retriever, reranker, router, tool caller, SQL generator, citation step)		 02 · Offline Evals
4. Grading method • Deterministic code
• Statistical/IR math
• Small ML classifier
• LLM-as-a-judge
• Human review		 LLM-as-a-Judge
5. Reference data • Reference-based (compares against a known-correct answer)
• Reference-free (grades using only question + context + output)		 End-to-End RAG Evals
6. Guardrail action • Blocking (allow, deny, modify, escalate)
• Non-blocking (flag and log)
• Sync vs. Async		 Guardrails Architecture

The central rule: A single check (such as "Is every claim supported by retrieved context?") can run as an offline eval in CI, an online monitor on 5% of live traffic, and a blocking guardrail before returning high-risk answers. Only the latency budget and failure action change.

Full table of contents

00 · Foundations

Guide What you will learn
01 · Why Evals and Guardrails Why demos fail in production, how evals and guardrails share checks, and the Analyze → Measure → Improve loop
02 · The Evals & Guardrails Map The five axes, master lookup tables, and Python code showing one check deployed three ways
03 · Agentic RAG Failure Modes Stage-by-stage failures (query, routing, retrieval, generation, citations, tools, SQL, security) and how to catch each one
04 · Glossary Plain-English definitions for ~90 terms used across this repository

01 · Datasets

Guide What you will learn
01 · Dataset Strategy Dataset sources, JSON schemas per eval type, slice coverage, and sample-size math
02 · SME Golden Datasets How to work with domain experts, write clear rubrics, and measure reviewer agreement
03 · Synthetic Datasets How to generate QA, multi-hop, tool, SQL, and adversarial test cases, and filter out bad rows
04 · Production Data Flywheel How to turn live traces and user feedback into versioned, PII-scrubbed test datasets

02 · Offline Evals

Guide What you will learn
01 · End-to-End RAG Evals The RAG triad (context relevance, faithfulness, answer relevancy), answer correctness, and context recall
02 · Retrieval Metrics Precision@k, Recall@k, F1@k, Hit Rate@k, MRR, MAP, and nDCG@k with Python implementations
03 · Generation Quality & Safety Toxicity, bias, tone, conciseness, refusal accuracy, and hallucination checks
04 · Agent, Tool & Trajectory Evals Tool selection, argument schemas, step trajectories, task completion, and multi-turn conversations
05 · Text-to-SQL Evals Execution accuracy, SQL component matching, syntax checks, and read-only safety rules
06 · Instruction & Citation Adherence Format constraints, JSON schema checks, and citation precision/recall
07 · LLM-as-a-Judge How to write binary Pass/Fail rubrics and calibrate LLM judges against human labels
08 · Running Offline Experiments Baselines, CI regression gates, statistical significance, and ablations

03 · Online Evals & Observability

Guide What you will learn
01 · Online Evals Running deterministic checks and sampled LLM judges on live traffic; shadow, canary, and A/B testing
02 · Observability & Tracing What spans to log, OpenTelemetry GenAI conventions, and trace privacy
03 · User Feedback & Implicit Signals Combining thumbs-up/down clicks with implicit signals like retries, edits, and escalations
04 · Monitoring, Alerting & Drift Dashboards, SLOs, drift detection, and an incident response runbook

04 · Guardrails

Guide What you will learn
01 · Guardrails Architecture Defense-in-depth layers, the five guardrail actions, latency budgets, and fail-open vs. fail-closed rules
02 · Input Guardrails Stopping direct prompt injection, jailbreaks, PII in prompts, and out-of-scope topics
03 · Retrieval & Context Guardrails Pre-retrieval RBAC filters, indirect injection scanning, and minimum relevance cutoffs
04 · Output Guardrails Checking faithfulness, verifying citations, redacting leaked PII, and validating JSON schemas
05 · Tool & Action Guardrails Least-privilege tools, human-in-the-loop confirmation, step budgets, SQL parsers, and MCP safety
06 · Access Control & RBAC Propagating user identity across retrieval, SQL row-level security, tools, and caches
07 · Threat Model & OWASP OWASP LLM Top 10 (2025), NIST AI RMF, MITRE ATLAS, and red-teaming workflows

05 · Decision Framework

Guide What you will learn
01 · Eval Selection Decision Tree Step-by-step trees to pick the right metrics for your architecture and stage
02 · Guardrail Selection Decision Tree Risk tiers (R0–R3) and which guardrails must run synchronously vs. asynchronously
03 · Metric Cheatsheet Every metric and guardrail check summarized in one reference table
04 · Implementation Playbook A 7-phase rollout plan from day zero to production, with RACI tables and launch checklists
05 · Maturity Model Levels 0 to 4 with a self-assessment scorecard for your team

06 · Tooling Landscape

Guide What you will learn
01 · Eval & Observability Tools Side-by-side comparison of RAGAS, DeepEval, TruLens, promptfoo, Arize Phoenix, Langfuse, LangSmith, Braintrust, and more
02 · Guardrail Tools Side-by-side comparison of NeMo Guardrails, Guardrails AI, Llama Guard, Presidio, Lakera, and cloud provider guardrails

07 · References

Curated papers, practitioner essays, security standards, benchmarks, and courses.

The running example

To keep every concept concrete, all guides use the same scenario:

Acme Internal Assistant, an enterprise agent that:

  1. Answers document questions over company policies and wikis using vector + keyword search, reranking, and citations.
  2. Answers data questions by generating SQL against a data warehouse (text-to-SQL).
  3. Takes actions via tools (such as create_ticket and lookup_employee).
  4. Serves multiple user roles (Employee, Manager, HR, Finance, Admin), enforcing strict RBAC on every document, table, and tool call.

How these guides are written

  • Quick orientation: Every page starts with a 3–4 bullet TL;DR and ends with Key takeaways and Further reading.
  • Consistent metric cards: Metrics are explained using a standard card: What it measures · Inputs required · How it works · Score range · Common pitfalls · When to use.
  • Calibrate your own thresholds: Sample sizes and score thresholds in this repo are starting rules of thumb. Always calibrate them on your own data.