Guardrails Architecture¶

TL;DR

• Inspect single requests at runtime. A guardrail checks an input, chunk, tool call, or output and decides to allow, block, modify, escalate, or flag it.
• Build layered defense-in-depth. Combine independent rules, classifiers, and deterministic authorization across layers because no single detector catches everything.
• Treat each guardrail as a production classifier. Define a spec, a latency budget, a fail-open or fail-closed mode, and should-block and should-allow test sets.
• Keep policy in versioned configuration. Separate detector scores from action thresholds and roll out new guardrails in three stages: shadow, then flag, then block.

• You will learn:
  • Guardrails vs evals
  • Layered defense model
  • Five guardrail actions
  • Sync, async, and streaming patterns
  • Detector families
  • Policy-as-config
  • Refusal UX
  • Guardrail metrics and spec template
• Prerequisites: Why evals and guardrails, The evals and guardrails map
• Related: Input guardrails, Retrieval & context guardrails, Output guardrails, Tool & action guardrails, Access control / RBAC, Threat model / OWASP

1. What a guardrail is (and is not)¶

A guardrail is a runtime check that inspects a specific artifact from a single interaction. It returns a decision that your system enforces immediately.

guardrail(artifact, context) -> Decision(action, reasons, scores, modified_artifact?)

• Artifact: the user message, a retrieved chunk, a planned tool call and its arguments, a generated SQL query, the final answer, or a streamed chunk.
• Context: user identity (role, tenant, entitlements), conversation state, policy version, and active tool scope.
• Decision: one of the five actions plus machine-readable reasons for logging, measurement, and debugging.

Guardrails vs evals¶

Guardrails and evals often use the same underlying detectors, such as a toxicity classifier. However, they serve different goals and have different engineering constraints.

Follow two core rules of thumb:

1. Evaluate every guardrail before deployment. A guardrail is a classifier. Measure its precision, recall, and latency before placing it in the user path (§9).
2. Do not turn every eval into a guardrail. Put a check inline only when a single miss causes more harm than the added latency, cost, and false positives. Otherwise, monitor the metric asynchronously with online evals.

What guardrails cannot do¶

• Guardrails do not replace authorization. Enforce document access with deterministic access control, not by prompting an LLM. See 06-access-control-rbac.md.
• Guardrails do not solve prompt injection. Detectors reduce attack success, while architecture bounds the damage when detectors miss. Use least privilege, untrusted-data isolation, and human approval for side effects. See 07-threat-model-owasp.md.
• Guardrails do not fix a weak base system. If 20% of answers are ungrounded, a groundedness rail will simply refuse 20% of requests. Fix your retriever and generator with offline evals first.

2. The layered (defense-in-depth) model¶

Each layer inspects a different artifact and catches failures that other layers cannot see. A request passes through several independent checks. Bad requests rarely slip through all layers when you mix different detector types with uncorrelated failure modes.

flowchart TD
    U["User message"] --> IN["Input rails: normalize, limits, injection, scope, PII, SQL-in-input"]
    IN -->|allow / modify| RT["Routing / intent: which pipeline, which tools are in scope"]
    IN -->|block| RF["Refusal / safe response"]
    RT --> RET["Retrieval and context rails: ACL filter, provenance, injection scan, relevance threshold"]
    RET --> LLM["LLM / agent loop"]
    LLM --> TA["Tool and action rails: arg validation, allowlists, SQL checks, approvals, budgets"]
    TA -->|tool result| TR["Tool-result rails: treat as untrusted data"]
    TR --> LLM
    TA -->|needs approval| HITL["Human in the loop"]
    LLM --> OUT["Output rails: groundedness, citations, leakage, toxicity, schema, policy"]
    OUT -->|allow / modify| ANS["Answer to user"]
    OUT -->|block| FB["Regenerate / fallback / refuse"]

    POL["Policy and authorization layer: identity, roles, entitlements, policy config"]
    POL -.-> RT
    POL -.-> RET
    POL -.-> TA
    POL -.-> OUT

    ASYNC["Async / non-blocking rails: logging, flagging, sampled judges, alerts"]
    IN -.-> ASYNC
    OUT -.-> ASYNC
    TA -.-> ASYNC

Design principles for the layered model

• Push checks as early as possible. Block out-of-scope requests at input before paying for retrieval and generation. Wait until later stages for checks that require downstream context, such as answer groundedness.
• Enforce authorization at the data and tool boundary. Never rely on the prompt for access control. The LLM must never retrieve a document that the user cannot read.
• Treat all external text as untrusted. User inputs, retrieved chunks, tool outputs, and web pages can all contain adversarial instructions. See 03.
• Diversify detector families. A regex, an ML classifier, and an LLM judge fail in different ways. Two LLM judges built on the same base model share similar blind spots.
• Make every decision observable. Record the decision, rail name, policy version, scores, and latency as span attributes on the trace (observability).

How framework rail types map to these layers. NeMo Guardrails names five rail types. Use this map when you read its docs:

Dialog rails are written in Colang. Two syntaxes exist (Colang 1.0 and 2.0), and NeMo ships a converter between them (migration guide), so check which one an example uses before you copy it. No framework rail type replaces the policy and authorization layer. Enforce it in your own data and tool code (06).

3. Guardrail actions¶

A guardrail does more than return yes or no. Choose the least disruptive action that mitigates the risk.

Action guidelines

• Use Modify more often. Redacting a credit card number is usually better than refusing the entire request. Rewriting a markdown image link to plain text is better than blocking a good answer.
• Pair Escalate with a real queue and SLA. Escalating to an unmonitored queue creates a silent block. When asking the user to confirm, show the exact tool, arguments, and impact rather than a model paraphrase.
• Roll out new guardrails in stages. Use three stages: shadow (log the verdict only; no user or reviewer effect), then flag (annotate the trace and notify reviewers; still no block), then block (enforce). Measure precision on real traffic at each stage before you promote the rail.
• Avoid dead-end blocks. Pair every block with a helpful fallback, such as a narrower answer, a documentation link, or a support escalation path.

4. Blocking vs non-blocking, sync vs async¶

Keep these two independent dimensions separate:

• Blocking vs non-blocking defines whether the guardrail can alter the current request (block, modify, or escalate vs flag only).
• Sync vs async defines whether the request waits for the guardrail before continuing (in-path vs parallel or post-hoc).

Rule of thumb for placement (heuristic):

5. Latency budgets, parallelization, streaming, and failure modes¶

5.1 Set a budget per layer¶

Users experience total end-to-end latency. Decide up front how much latency budget your guardrails can consume. The table below shows heuristic starting points for an enterprise assistant (measure your own system):

Track p95 and p99 latency rather than the mean. LLM-based guardrails have long latency tails.

5.2 Run independent rails in parallel, and in parallel with the main work¶

Most input guardrails do not depend on each other, and retrieval does not depend on their verdict. Run input guardrails and retrieval concurrently. Gate only the expensive LLM call on the guardrail results.

sequenceDiagram
    participant U as User
    participant O as Orchestrator
    participant IR as Input rails
    participant R as Retriever
    participant L as LLM
    participant OR as Output rails
    U->>O: message
    par input rails in parallel
        O->>IR: injection, scope, PII, toxicity
    and retrieval speculatively
        O->>R: ACL-filtered search
    end
    IR-->>O: verdicts
    R-->>O: chunks
    alt any blocking rail failed
        O-->>U: refusal or fallback, discard retrieval
    else all passed
        O->>L: prompt with context
        L-->>O: draft answer
        O->>OR: groundedness, citations, leakage
        OR-->>O: verdict
        O-->>U: answer or fallback
    end

Keep three caveats in mind:

• Ensure speculative work is side-effect free. Read-only retrieval is safe to run speculatively. Never run tool calls with side effects speculatively.
• Enforce ACLs inside speculative retrieval. Permission filtering is part of the search query itself, not a parallel rail that you can discard later.
• Handle input modifications carefully. If an input rail redacts PII, your speculative retrieval used the unredacted text. Ensure the LLM and logs receive only the redacted text, or re-run retrieval when needed.

Here is a pure-Python sketch of parallel guardrails with per-rail timeouts and failure modes:

import asyncio
from dataclasses import dataclass, field
from enum import Enum
from typing import Awaitable, Callable

class Action(str, Enum):
    ALLOW = "allow"
    BLOCK = "block"
    MODIFY = "modify"
    ESCALATE = "escalate"
    FLAG = "flag"

@dataclass
class Decision:
    rail: str
    action: Action
    reasons: list[str] = field(default_factory=list)
    score: float | None = None
    modified_text: str | None = None

@dataclass
class Rail:
    name: str
    check: Callable[[str], Awaitable[Decision]]
    timeout_s: float
    fail_closed: bool  # what to do on timeout/error

async def run_rail(rail: Rail, text: str) -> Decision:
    try:
        return await asyncio.wait_for(rail.check(text), timeout=rail.timeout_s)
    except Exception as exc:  # includes asyncio.TimeoutError
        action = Action.BLOCK if rail.fail_closed else Action.FLAG
        return Decision(rail.name, action, [f"rail_error:{type(exc).__name__}"])

SEVERITY = {Action.BLOCK: 4, Action.ESCALATE: 3, Action.MODIFY: 2, Action.FLAG: 1, Action.ALLOW: 0}

async def run_input_rails(text: str, rails: list[Rail]) -> tuple[Action, list[Decision]]:
    decisions = await asyncio.gather(*(run_rail(r, text) for r in rails))
    final = max((d.action for d in decisions), key=SEVERITY.__getitem__, default=Action.ALLOW)
    return final, list(decisions)

Use the most severe action wins rule to combine verdicts (block > escalate > modify > flag > allow). When multiple guardrails modify the text, apply them sequentially in a fixed order: normalization → secret redaction → PII redaction.

5.3 Short-circuit cheap-to-expensive (cascades)¶

When you run guardrails sequentially, order them by cost and trigger frequency:

1. Run deterministic, near-free checks first: length limits, normalization, regex denylists, allowlists, and schema checks.
2. Run small ML classifiers second: models that take tens of milliseconds per call (for example, Meta reports about 19–92 ms per 512-token Prompt Guard 2 call on an A100 GPU). Benchmark on your own hardware; CPU latency is usually higher.
3. Run LLM-based rails last: invoke them only for remaining ambiguous cases.

A cascade uses a cheap detector's confidence score to decide whether to invoke an expensive detector. Resolve high-confidence allows and blocks with the cheap classifier, and route only the gray zone to the LLM judge. This pattern cuts LLM-rail volume sharply while preserving accuracy (validate the threshold on your dataset).

5.4 Streaming output and chunk-level checking¶

Streaming lowers time-to-first-token, but you cannot un-show tokens once the client renders them.

Here is a chunk-level hold-back pattern in pure Python:

import re
from typing import Iterable, Iterator

SECRET = re.compile(r"AKIA[0-9A-Z]{16}|sk-[A-Za-z0-9_-]{20,}")  # illustrative; use your full secrets ruleset

def _redact_closed(buf: str) -> str:
    # Only redact matches that are followed by more text; a match touching the end may still grow.
    return SECRET.sub(lambda m: "[REDACTED]" if m.end() < len(buf) else m.group(0), buf)

def guarded_stream(tokens: Iterable[str], holdback_chars: int = 64) -> Iterator[str]:
    """Release text only once it is `holdback_chars` behind the head, and never release
    part of a match that may still be growing."""
    buf = ""
    for tok in tokens:
        buf = _redact_closed(buf + tok)
        cut = len(buf) - holdback_chars
        open_match = next((m for m in SECRET.finditer(buf) if m.end() == len(buf)), None)
        if open_match:
            cut = min(cut, open_match.start())
        if cut > 0:
            yield buf[:cut]
            buf = buf[cut:]
    yield SECRET.sub("[REDACTED]", buf)

The character class includes _ and - because current OpenAI keys look like sk-proj-…; a class without them stops the match at the first hyphen and leaks the rest of the key. Set the hold-back window at least as long as the longest pattern you must catch. Hold any match touching the end of the buffer until it finishes growing so secrets cannot leak a few characters at a time. Frameworks like NeMo Guardrails provide built-in streaming output rails. See guardrail tools.

5.5 Fail-open vs fail-closed¶

Every guardrail will eventually time out, throw an error, or hit a rate limit. Choose your failure mode in advance:

• Fail-closed: treat any guardrail error as a block. This maximizes safety, but total availability becomes the product of every guardrail's availability.
• Fail-open: treat any guardrail error as an allow and flag it. This preserves availability, but attackers can bypass the guardrail if they can force a timeout.

Rule of thumb. Fail closed for tool-enabled turns, high-stakes intents, and every access-control check. Fail open (and flag) only for read-only, low-risk Q&A.

Make fail-closed practical by degrading gracefully (fall back from LLM rail → classifier → regex). Use circuit breakers, per-rail timeouts tighter than the request timeout, and strict input length caps so attackers cannot force timeouts.

6. Detector families: rules vs ML classifiers vs LLM-based rails¶

Recommended defaults

• Use deterministic controls wherever the policy is crisp. Enforce permissions, schemas, allowlists, and read-only SQL in code, never with a model.
• Use small ML classifiers for high-volume screening across all incoming requests.
• Reserve LLM rails for the gray zone in a cascade or for low-volume, high-stakes actions like write approvals.
• Prefer allowlists over denylists. Allowlisting specific tables, domains, and intents is robust. Denylisting forbidden words is an endless game of whack-a-mole.

7. Guardrail policy as configuration¶

Hard-coding thresholds and actions in application code makes policies hard to audit, tune, and roll back. Store your guardrail policy as versioned configuration. Review policy changes like code, evaluate them in CI before rollout, and stamp the active policy_version on every trace.

# guardrails/policy.yaml: illustrative schema, not tied to any framework
version: "2026-10-01.3"
defaults:
  on_error: fail_closed        # override to fail_open_flag only for read-only, low-risk Q&A rails
  timeout_ms: 300
  log_scores: true

rails:
  - name: prompt_injection_input
    layer: input
    detector: { type: classifier, model: prompt-guard, version: "2" }
    threshold: 0.90            # heuristic start; tune on your test set
    action: block
    stage: block               # shadow | flag | block
    on_error: fail_closed      # or route the turn to a no-tools mode
    applies_to: { pipelines: [agentic_tools, text_to_sql] }

  - name: prompt_injection_input_qa
    layer: input
    detector: { type: classifier, model: prompt-guard, version: "2" }
    threshold: 0.90
    action: flag               # read-only, low-risk Q&A: flag, do not block
    stage: block               # the stage controls whether the action takes effect
    on_error: fail_open_flag
    applies_to: { pipelines: [doc_qa] }

  - name: pii_input
    layer: input
    detector: { type: presidio, entities: [EMAIL_ADDRESS, PHONE_NUMBER, US_SSN, CREDIT_CARD] }
    threshold: { default: 0.6, PHONE_NUMBER: 0.4, US_SSN: 0.5, PERSON: 0.7 }   # per entity; calibrate
    action: { type: modify, method: pseudonymize }
    stage: block

  - name: groundedness_output_high_stakes
    layer: output
    detector: { type: nli, model: hhem }
    threshold: 0.5
    action: { type: regenerate, max_attempts: 1, then: abstain_with_sources }
    stage: block
    on_error: fail_closed      # HR, finance, security intents
    applies_to: { intents: [hr_policy, finance, security_procedure] }

  - name: groundedness_output
    layer: output
    detector: { type: nli, model: hhem }
    threshold: 0.5
    action: { type: regenerate, max_attempts: 1, then: abstain }
    stage: block
    on_error: fail_open_flag   # read-only, low-risk Q&A only
    applies_to: { intents: [general_policy_qa] }

overrides:
  - match: { role: finance }
    rails: { pii_input: { entities_exempt: [CREDIT_CARD] } }   # if policy allows; document why

Configuration best practices

• Separate detection from decision. Have detectors emit raw scores, and use the policy config to map scores to actions per pipeline, role, tenant, or region.
• Roll out in stages. Deploy every new guardrail or threshold with stage: shadow (log only), promote it to stage: flag (annotate and notify reviewers), and switch to stage: block only after you have measured false positives on live traffic.
• Version and stamp every trace. Log policy_version and per-rail decisions on every trace so you can explain refusals and replay traffic against candidate policies.
• Run regression tests on policy changes. Trigger your should-block and should-allow eval suites in CI whenever policy.yaml changes (running offline experiments).
• Review per-role overrides explicitly. Unreviewed overrides silently weaken your security posture.
• Treat guardrail models as supply chain. A guardrail model is a dependency that sits in the security path (OWASP LLM04:2026 Supply Chain). Pin the exact model revision (for Hugging Face, a commit hash, not main), load safetensors weights instead of pickle files (Hugging Face pickle security), mirror the weights internally, and re-run the full should-block and should-allow suites before you accept any upgrade.
• Compare framework support. Tools like NeMo Guardrails, Guardrails AI, LLM Guard (now archived), AWS Bedrock Guardrails, and the OpenAI Agents SDK implement parts of this pattern. Compare them in guardrail tools.

8. Refusal UX and messaging¶

A guardrail's user-facing message is part of your product UX. Poor refusal messages frustrate legitimate users and push them to work around the system.

Refusal UX principles

1. State what you cannot do and offer a next step. For example: "I can't help with salary data for other employees. For your own pay, check Workday → Pay or contact HR."
2. Hide detector internals. Showing "Prompt injection score 0.93 from classifier X" helps attackers tune their payloads. Log the details internally and show a generic category to the user.
3. Never accuse the user. False positives happen. Tell the user what the assistant can support instead of saying "This looks like an attack."
4. Separate can't from don't know. Use separate templates for policy refusals ("I'm not able to share that") and abstentions ("I couldn't find this in documents you can access").
5. Never confirm the existence of restricted files. Saying "That document is restricted" leaks its existence. Say "I couldn't find anything you have access to on that."
6. Prefer partial help over total refusal. Answer the allowed portion of a multi-part question and decline only the restricted part.
7. Provide a user feedback button. A "This shouldn't have been blocked" button collects labeled false positives automatically (user feedback).
8. Use static templates per reason category. Never let the LLM improvise refusal messages, which can leak context or be manipulated.
9. Do not turn the guardrail into an oracle. If each rail returns a different message, or a block returns faster than an allow, an attacker can probe which rail fired and tune a payload offline. Show the same generic message for all security blocks (injection, leakage, unsafe content). Keep specific reason codes such as INJ-02 or LEAK-CANARY in internal logs only. Consider rate limits per user on repeated blocks.

9. Measuring guardrails themselves¶

A guardrail is a binary classifier that predicts should block vs should allow. Evaluate it offline on labeled test sets and online on sampled production traffic.

9.1 Test sets¶

• Should-block set (attacks and policy violations): collect known jailbreaks, prompt injections, PII samples, out-of-scope queries, and poisoned documents. Include obfuscated variants such as Base64 encoding, typos, and non-English translations.
• Should-allow set (benign lookalikes): collect legitimate queries that look risky. Include security questions on SQL injection, HR questions on harassment policy, and prompts containing "ignore". This dataset prevents over-blocking.
• Realistic base-rate sample: maintain a representative sample of ordinary production traffic (mostly benign) to estimate your true production false-positive rate.

Build and maintain these datasets using the workflows in dataset strategy and the production data flywheel.

9.2 Metrics¶

                     Rail says BLOCK     Rail says ALLOW
Should block              TP                  FN   (miss / bypass)
Should allow              FP (over-block)     TN

precision          = TP / (TP + FP)      # of blocks, how many were right
recall (catch rate)= TP / (TP + FN)      # of bad items, how many were caught
false positive rate= FP / (FP + TN)      # of good items, how many were wrongly blocked
false negative rate= FN / (FN + TP) = 1 - recall
F_beta             = (1 + b^2) * P * R / (b^2 * P + R)   # b > 1 favours recall

def guardrail_metrics(y_true: list[bool], y_block: list[bool]) -> dict[str, float]:
    """y_true: item should be blocked; y_block: rail blocked it."""
    tp = sum(t and b for t, b in zip(y_true, y_block))
    fp = sum((not t) and b for t, b in zip(y_true, y_block))
    fn = sum(t and (not b) for t, b in zip(y_true, y_block))
    tn = sum((not t) and (not b) for t, b in zip(y_true, y_block))
    safe = lambda n, d: n / d if d else float("nan")
    return {
        "precision": safe(tp, tp + fp),
        "recall": safe(tp, tp + fn),
        "fpr": safe(fp, fp + tn),
        "fnr": safe(fn, fn + tp),
    }

Account for base rates. Suppose only 0.1% of real traffic is malicious. A guardrail with 99% recall and a 1% false-positive rate will block ~10 legitimate users for every attacker it catches. Always estimate precision at your realistic production base rate, not on a 50/50 balanced test set.

9.3 Beyond accuracy¶

9.4 Choosing thresholds¶

1. Sweep thresholds across your test sets and plot recall against false-positive rate (ROC) or precision against recall.
2. Optimize for error cost rather than F1. For irreversible harm, set a minimum recall target and accept a higher FPR. For low-stakes annoyance, cap FPR (for example, ≤ 0.5–1% of benign traffic) and accept lower recall.
3. Use two thresholds for cascades. Block scores above t_block, route scores between t_flag and t_block to an LLM guardrail or flag queue, and allow scores below t_flag.
4. Re-calibrate thresholds whenever you upgrade the detector model, change the main generator, or observe a shift in user traffic.

10. The guardrail spec template¶

Write a one-page specification before building any guardrail. Answering these questions up front prevents accidental policy decisions in production.

### Guardrail spec: <name>

| Field | Value |
|---|---|
| **Name / ID** | e.g. `sql_in_input` |
| **Purpose / risk addressed** | What harm it prevents; link to threat model entry (07) |
| **Layer** | input / routing / retrieval / tool / output / policy |
| **Trigger / scope** | Which pipelines, intents, roles, tenants, tools it runs on |
| **Detector** | rule / classifier / LLM; model + version; features used |
| **Action** | allow / block / modify (redact, mask, rewrite) / escalate / flag |
| **Mode** | sync-blocking / async-non-blocking / optimistic-parallel; rollout stage `shadow` / `flag` / `block` |
| **Threshold(s)** | Score cut-offs and how they were chosen |
| **Failure mode** | fail-open or fail-closed; timeout; fallback detector |
| **Latency budget** | p95 target for this rail |
| **User-facing message** | Template ID and text; next step offered |
| **Owner** | Team/person accountable for tuning and incidents |
| **Test set** | Should-block and should-allow datasets (location, size, last refreshed) |
| **Metrics & targets** | precision, recall, FPR on benign traffic, latency p95, cost |
| **Monitoring & alerts** | Block-rate band, error rate, dashboards |
| **Review cadence** | e.g. monthly threshold review, after model upgrades |
| **Policy version** | Config version where this spec is active |

Filled example: SQL-in-input for the enterprise assistant

11. Implementation checklist¶

• Write a threat model and map every guardrail to a specific threat (07).
• Enforce authorization deterministically at retrieval and tool boundaries, never via prompts (06).
• Create a spec (§10), a should-block test set, and a should-allow test set for every guardrail.
• Define an explicit fail-open or fail-closed mode and a tight timeout for each guardrail. Fail closed for tool-enabled turns, high-stakes intents, and access control.
• Run independent input guardrails in parallel with each other and with side-effect-free retrieval.
• Order sequential guardrails from cheap to expensive, and reserve LLM rails for the gray zone.
• Choose an explicit streaming strategy for every output guardrail (buffer, chunk hold-back, or optimistic).
• Store guardrail policy in versioned config and log policy_version and rail verdicts on every trace.
• Roll out new guardrails through shadow → flag → block, and review flagged traffic before you enforce blocks.
• Pin guardrail model revisions, load safetensors weights, and re-run guardrail evals on every detector upgrade.
• Use one generic message for all security blocks so the guardrail does not become an oracle.
• Write templated, non-accusatory refusal messages that hide detector internals and offer a next step.
• Build dashboards and anomaly alerts for block rates, latency overhead, timeouts, and token cost.
• Run regular red-team exercises and add every bypass and false positive to your regression test sets.

Key takeaways¶

• Distinguish guardrails from evals. Guardrails make per-request runtime decisions, while evals measure aggregate quality. Every guardrail needs its own eval.
• Layer defenses across the pipeline. Combine input, retrieval, tool, and output guardrails over a deterministic authorization foundation.
• Choose the least disruptive action. Prefer modifying or escalating over blocking when safe, and roll out new guardrails through shadow, flag, then block.
• Separate blocking behavior from execution mode. Keep irreversible-harm checks, tool-enabled turns, and high-stakes intents synchronous and fail-closed. Fail open (and flag) only for read-only, low-risk Q&A.
• Protect user-facing latency. Use parallel execution, cheap-to-expensive cascades, and chunk-level streaming buffers, and track p95 latency.
• Match the detector family to the problem. Use deterministic rules and allowlists for crisp policies, small classifiers for volume, and LLM judges for ambiguous gray zones.
• Measure over-blocking rigorously. Maintain a realistic should-allow dataset and evaluate false-positive rates at production base rates.

Further reading¶

• OWASP, Top 10 for LLM Applications 2026 (mapping to the 2025 IDs in 07) and the LLM Prompt Injection Prevention Cheat Sheet
• NIST, AI Risk Management Framework
• Rebedea et al., NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails · docs
• Inan et al., Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations · Llama Guard 4 model card
• NVIDIA, NeMo Guardrails rail types
• Chennabasappa et al., LlamaFirewall: An open source guardrail system for building secure AI agents
• OpenAI Cookbook, How to implement LLM guardrails (parallel guardrails, latency trade-offs)
• OpenAI Agents SDK, Guardrails
• AWS, Amazon Bedrock Guardrails
• Beurer-Kellner et al., Design Patterns for Securing LLM Agents against Prompt Injections
• Debenedetti et al., Defeating Prompt Injections by Design (CaMeL)
• In this repo:
  • Guardrail selection decision tree
  • Guardrail tools
  • Threat model / OWASP