Instruction & Citation Adherence Evals¶

TL;DR

• Treat your system prompt as a testable contract. Decompose complex system prompts into atomic instructions and assign a checker to each one.
• Run deterministic checks before calling an LLM judge. Validate length, format, JSON schemas, citation IDs, and exact quotes in code.
• Use binary judges or NLI models for semantic rules. Evaluate tone, policy compliance, and citation entailment with single-requirement binary checks.
• Measure both refusal recall and false-refusal rate. Test must-refuse inputs alongside benign look-alikes so safety rules do not block legitimate users.

• You will learn:
  • Decomposing system prompts into testable checks
  • IFEval-style verifiable instruction checks
  • JSON and Pydantic schema validation
  • Persona, policy, and refusal-policy evals
  • Citation format, coverage, ALCE recall/precision, and quote verification
  • Newer instruction-following and citation benchmarks, and the limits of automatic attribution
  • Deterministic checkers vs LLM judges
• Prerequisites: Evals & guardrails map, End-to-end RAG evals
• Related: LLM-as-a-judge, Generation quality & safety evals, Output guardrails, Agent tool & trajectory evals

1. Why these two belong together¶

Instruction following and citation checking are both adherence evaluations. They compare model output against an explicit contract rather than a single gold reference answer. This contract comes from four sources:

• System prompt and developer instructions: Rules like "answer in ≤ 150 words", "cite every claim as [doc:<id>]", or "never promise delivery dates".
• User requests: Constraints in the turn, such as "give me 3 bullet points" or "format this as a table".
• Tool and output schemas: Structural rules, such as validating every create_ticket payload against a strict schema.
• Organizational policies: Business rules, such as refusing legal advice, escalating HR complaints, or blocking salary queries from non-HR roles.

Consider our running example of an internal enterprise assistant over company docs, a data warehouse, and a ticketing tool. Its system prompt typically contains 10 to 30 distinct rules. Most teams test none of these rules explicitly and rely on manual spot-checks instead. Fix this systematically: decompose your system prompt into atomic, testable instructions and assign each one a checker.

flowchart LR
    SP["System prompt + policies + schemas"] --> D["Decompose into atomic instructions"]
    D --> V{"Verifiable by code?"}
    V -->|Yes| C["Deterministic checker: regex, counter, JSON schema, substring"]
    V -->|No| J["Binary LLM judge per instruction"]
    C --> R["Per-instruction pass rates"]
    J --> R
    R --> A["Prompt-level and instruction-level accuracy"]

2. Instruction following¶

2.1 Decomposing a system prompt¶

Rule of thumb: If you cannot write a checker for an instruction, the instruction is too vague for the model to follow reliably. Rewrite the instruction in your prompt until you can test it.

2.2 Verifiable instruction checks (IFEval-style)¶

• What it measures: Compliance with instructions that you can verify deterministically in code. IFEval (Zhou et al., 2023) popularized this across 25 instruction types, including length, keywords, JSON format, bullet counts, casing, punctuation, and language.
• Inputs required: Model output and the list of parameterized instructions. You do not need a reference answer.

• How to compute it: Run each instruction checker and aggregate the results in two ways:

  instruction_level_acc = (# instructions followed) / (# instructions across all prompts)
  prompt_level_acc      = (# prompts where ALL instructions followed) / (# prompts)
  

  IFEval also computes strict and loose scores. The loose score builds 8 variants of each response: the original, the response without * characters, without its first line, without its last line, without both, and the same three line removals on the *-free text. An instruction passes in loose mode if any variant passes. This reduces false negatives from markdown emphasis and from preambles such as "Sure, here is...".

• Score range & interpretation: 0 to 1. Use prompt-level accuracy as your main user-facing metric because a single violated rule fails the turn. Use instruction-level accuracy to diagnose which specific rule needs prompt work.

• Pitfalls / gotchas: Word counting varies across tokenizers and languages without spaces (such as Chinese or Japanese). Simple keyword checks are vulnerable to keyword stuffing. Conditional instructions ("≤ 150 words unless the user asks for detail") require encoding the condition in your test metadata.
• When to use / when not to: Always use deterministic checks for code-verifiable rules. Reuse them as online monitors or blocking output guardrails.
• Implemented in: google-research/instruction_following_eval, promptfoo assertions (contains, icontains, regex, is-json, contains-json, javascript, python), or custom Python functions.

# Requires: pip install regex  (the stdlib re module has no Unicode emoji properties)
import re
from dataclasses import dataclass
from typing import Callable

import regex

@dataclass
class Instruction:
    id: str
    check: Callable[[str], bool]

def max_words(n):        return lambda out: len(out.split()) <= n
def must_include(*kws):  return lambda out: all(k.lower() in out.lower() for k in kws)
# (?<!\w)...(?!\w) instead of \b, so terms like "C++" or ".NET" still match.
def must_avoid(*terms):  return lambda out: not any(re.search(rf"(?<!\w){re.escape(t)}(?!\w)", out, re.I) for t in terms)
def min_bullets(n):      return lambda out: len(re.findall(r"^\s*(?:[-*•]|\d+\.)\s+", out, re.M)) >= n
# Accepts "Sources", "Sources:", "## Sources", "**Sources:**", "**Sources**:".
def has_section(title):  return lambda out: re.search(
    rf"^\s*#*\s*(?:\*\*|__)?{re.escape(title)}:?(?:\*\*|__)?:?\s*$", out, re.M | re.I) is not None
# Emoji presentation, text symbols forced to emoji with U+FE0F, and keycaps. Flags (🇩🇪) match; "✓" does not.
EMOJI = regex.compile(r"\p{Emoji_Presentation}|\p{Extended_Pictographic}\uFE0F|[0-9#*]\uFE0F?\u20E3")
def no_emoji():          return lambda out: not EMOJI.search(out)

def evaluate(output: str, instructions: list[Instruction]) -> dict:
    results = {i.id: bool(i.check(output)) for i in instructions}
    return {"per_instruction": results, "prompt_pass": all(results.values())}

def aggregate(runs: list[dict]) -> dict:
    n_instr = sum(len(r["per_instruction"]) for r in runs)
    n_followed = sum(sum(r["per_instruction"].values()) for r in runs)
    return {"instruction_level_acc": n_followed / n_instr,
            "prompt_level_acc": sum(r["prompt_pass"] for r in runs) / len(runs)}

SYSTEM_RULES = [
    Instruction("I1_max_150_words", max_words(150)),
    Instruction("I5_no_codenames", must_avoid("Project Falcon", "BLUEJAY")),
    Instruction("I6_sources_section", has_section("Sources")),
    Instruction("I7_no_emoji", no_emoji()),
]
evaluate("The PTO policy allows 25 days [doc:hr-12].\n\nSources:\n- [doc:hr-12]", SYSTEM_RULES)
# {'per_instruction': {... all True}, 'prompt_pass': True}

Beyond IFEval. The IFBench authors found that most models overfit the small set of constraints used in benchmarks such as IFEval. Use newer public sets to check that instruction following generalizes before you trust a vendor claim:

Your own system-prompt rules remain the primary test set. Public benchmarks only help you shortlist models.

Language adherence: Check response language with a language-ID model such as fastText lid.176 or langdetect. Short strings and code-mixed text often misclassify, so enforce a minimum character length and treat low-confidence predictions as "unknown" rather than "fail".

2.3 Non-verifiable instructions: decomposed binary judges¶

• What it measures: Compliance with semantic instructions that require reading comprehension ("use bullet points for lists", "explain assumptions", "do not speculate").
• Inputs required: Atomic instruction, user input, and assistant output. You do not need a reference answer.

• How to compute it: Run one binary yes/no judge per instruction instead of a single 1–10 holistic score. InFoBench formalizes this as the Decomposed Requirements Following Ratio (DRFR), which evaluates each decomposed requirement separately and reports the fraction satisfied.

  You are checking ONE requirement.
  REQUIREMENT: "When listing 3 or more items, use bullet points."
  USER MESSAGE: {input}
  ASSISTANT RESPONSE: {output}
  Is the requirement applicable to this response? If not applicable, answer "n/a".
  If applicable, think step by step about whether it is satisfied, then answer.
  Output JSON: {"reasoning": "...", "verdict": "pass" | "fail" | "n/a"}
  

• Score range & interpretation: 0 to 1 per instruction. Exclude n/a verdicts from the denominator, but monitor your n/a rate; an instruction that is always n/a has zero test coverage in your dataset.

• Pitfalls / gotchas: Judges often mark inapplicable rules as fail unless you provide an explicit n/a option. Evaluating too many requirements in one prompt hurts accuracy, so batch at most 3 to 5 closely related checks. Validate every judge against human labels (see LLM-as-a-judge).
• When to use / when not to: Use binary judges for high-value instructions that code cannot verify. Skip rules that users do not care about.
• Implemented in: DeepEval PromptAlignmentMetric and GEval, promptfoo llm-rubric, or custom judge prompts.

2.4 System-prompt adherence over multi-turn conversations and under pressure¶

Single-turn tests miss common ways that instruction adherence degrades in production:

Use the framing from OpenAI's Instruction Hierarchy (Wallace et al., 2024) to design conflict tests. The paper ranks system messages above user messages, and both above model outputs and tool outputs. The separate developer level between system and user comes from the OpenAI Model Spec, not from the paper. Decide explicitly which system defaults a user may override (such as formatting preferences) and which rules are immutable (such as safety and RBAC policies).

Plot instruction adherence against turn index on multi-turn evaluations. A downward slope reveals that the model loses track of system instructions as the context window fills.

2.5 Persona, tone and policy adherence¶

Persona and tone

• What it measures: Whether the output matches your target voice ("professional, concise, no jokes, no emojis, second person").
• Inputs required: Persona specification and assistant output.
• How to compute it: Split the persona into verifiable checks (emojis, exclamation marks, reading level, banned phrases) and a binary style judge. Include concrete pass and fail examples in the judge prompt.
• Pitfalls / gotchas: Tone is subjective. Collect 20 to 30 SME-labelled examples before trusting a tone judge, and avoid vague 1–5 tone ratings.
• When to use / when not to: Prioritize tone evals for customer-facing or executive-facing assistants. Deprioritize them for internal engineering tools.

Policy adherence

• What it measures: Compliance with organizational rules (no legal or medical advice, no delivery-date promises, escalating harassment reports to HR).
• Inputs required: Policy text, user input, assistant output, and test prompts designed to trigger each policy alongside near-miss prompts that should not trigger it.
• How to compute it: Run a per-policy binary judge or classifier across your targeted test set and report the pass rate per policy.
• Pitfalls / gotchas: Policies tested only on obvious triggers will show 100% pass rates while failing in production. Include indirect phrasings, multi-turn build-ups, and role-play prompts. Enforce high-stakes policies as online output guardrails as well (see Output guardrails).

2.6 Refusal policy: over-refusal and under-refusal¶

A refusal policy defines what the assistant must decline (such as salary data for non-HR roles) and what it must answer. Evaluate refusal behavior as a binary classification problem.

• What it measures: Whether the agent refuses prohibited requests and answers permitted requests.
• Inputs required: User input, user role, assistant output, and a ground-truth label should_refuse ∈ {true, false}.

• How to compute it: Classify each response as a refusal or compliance using keyword heuristics or an LLM judge. Track partial compliance separately and build the confusion matrix:

  	Agent refused	Agent complied
  Should refuse	✅ correct refusal (TP)	❌ under-refusal / policy breach (FN)
  Should answer	❌ over-refusal (FP)	✅ helpful (TN)

  refusal_recall      = TP / (TP + FN)   # how many must-refuse cases were refused   → safety
  false_refusal_rate  = FP / (FP + TN)   # how many legit requests were blocked      → helpfulness
  

• Score range & interpretation: Both metrics range from 0 to 1. Track both metrics together, because optimizing one in isolation degrades the other. Under-refusal on RBAC-restricted data causes security incidents, whereas a high false-refusal rate drives users to abandon the assistant.

• Worked example: Suppose your dataset has 40 should-refuse items (salary queries from an employee role) and 160 should-answer items. The should-answer set includes benign look-alikes like "where is the salary band policy doc?" or "how do I kill a stuck Linux process?". If the agent refuses 38 of 40 forbidden requests and 12 of 160 benign requests, refusal_recall = 0.95 and false_refusal_rate = 0.075.
• Pitfalls / gotchas: You cannot measure over-refusal without benign look-alike prompts. Use XSTest (Röttger et al., 2023) as a template for crafting safe prompts that contain sensitive keywords. Also evaluate refusal helpfulness: check whether the refusal explains the policy and directs the user to an approved channel.
• When to use / when not to: Run refusal evals on every assistant that enforces access controls or content policies.
• Implemented in: Custom evaluators or judge frameworks. See Generation quality & safety evals and Access control & RBAC.

3. Format and schema adherence¶

Agents generate structured outputs on almost every turn, including tool-call arguments, UI JSON payloads, classification labels, and SQL queries. Format failures are easy to catch deterministically and critical to prevent because a parse failure crashes the turn.

3.1 Parse rate and schema validity¶

• What it measures: Whether the output parses and validates against your schema (required fields, types, enums, ranges, and regex patterns).
• Inputs required: Raw model output and target schema (JSON Schema or Pydantic model). You do not need ground truth.
• How to compute it: Compute parse_rate = parsed / total and schema_valid_rate = valid / total. Log specific validation error classes (missing field, wrong type, invalid enum, extra field).
• Score range & interpretation: 0 to 1. When you use native structured outputs or constrained decoding, syntactic validity should reach ~1.0, leaving only semantic field errors.
• Pitfalls / gotchas: Watch for markdown code fences wrapping JSON, trailing explanations, and overly permissive schemas (like str instead of Enum). Tighten your schema first; strict schemas improve both evaluation signal and runtime guardrails.
• When to use / when not to: Always validate every structured output and tool call.
• Implemented in: DeepEval JsonCorrectnessMetric, promptfoo is-json and contains-json, or pydantic and jsonschema directly.

3.2 Field-level correctness¶

A schema-valid payload can still contain wrong values. For "my laptop won't boot, urgent, I'm in Berlin", compare the generated create_ticket arguments against a gold payload field by field:

Report per-field accuracy to pinpoint which fields the model extracts poorly. To evaluate whether the agent selected the right tool in the first place, see Agent tool & trajectory evals.

import re
from enum import Enum
from pydantic import BaseModel, Field, ValidationError

class Priority(str, Enum):
    low = "low"; medium = "medium"; high = "high"

class CreateTicket(BaseModel):
    model_config = {"extra": "forbid"}            # unexpected fields are an error, not ignored
    category: str = Field(pattern=r"^(hardware|software|access|other)$")
    priority: Priority
    summary: str = Field(min_length=5, max_length=200)
    location: str | None = None

def schema_and_field_eval(raw_output: str, gold: dict, exact_fields=("category", "priority")) -> dict:
    cleaned = re.sub(r"^```[\w-]*\s*|\s*```$", "", raw_output.strip())   # any ```lang fence
    try:
        obj = CreateTicket.model_validate_json(cleaned)
    except ValidationError as e:
        return {"valid": False, "errors": [err["type"] + ":" + ".".join(map(str, err["loc"])) for err in e.errors()]}
    pred = obj.model_dump(mode="json")
    fields = {f: pred.get(f) == gold.get(f) for f in exact_fields}
    if gold.get("location") is not None:
        fields["location"] = (pred.get("location") or "").strip().lower() == gold["location"].strip().lower()
    return {"valid": True, "field_correct": fields}

schema_and_field_eval('{"category": "hardware", "priority": "urgent", "summary": "Laptop will not boot"}',
                      {"category": "hardware", "priority": "high"})
# {'valid': False, 'errors': ['enum:priority']}

4. Citation evals¶

Citations build user trust in enterprise assistants by letting users click through to verify HR policies or runbooks. Evaluate citations across four escalating questions plus quote verification:

flowchart TD
    A["Answer with citations"] --> B["1 Format validity: parseable and IDs exist in retrieved set?"]
    B --> C["2 Coverage: does every claim carry a citation?"]
    C --> D["3 Support: do cited passages entail the claim?"]
    D --> E["4 Precision: is each citation actually needed and relevant?"]
    A --> Q["Quote verification: quoted text is an exact substring of the source?"]
    B -.->|deterministic| X["cheap, run online"]
    D -.->|NLI or LLM judge| Y["costlier, sample online"]

4.1 Background: AIS and ALCE¶

• AIS (Rashkin et al., 2021): Statement s is attributable to source P if a reader agrees "According to P, s". AIS separates interpretability (whether the statement makes sense on its own) from attribution. Automated versions (AutoAIS, Bohnet et al., 2022) use an NLI model to test whether P entails s.
• ALCE (Gao et al., 2023): A benchmark and metric suite for citation generation. It measures fluency, correctness, citation recall, and citation precision using an NLI entailment model (TRUE, based on T5-11B). See princeton-nlp/ALCE.

Distinguish citation support from overall RAG faithfulness (see End-to-end RAG evals). Faithfulness checks whether the retrieved context as a whole supports the answer. Citation support checks whether the specific passage cited on a sentence supports that sentence. An answer can be 100% faithful to the context while attaching the wrong [doc:id] to every claim. Wrong citation links destroy user trust when users click through to verify.

4.2 Citation format validity¶

• What it measures: Whether citations follow the required syntax and reference document IDs that were actually retrieved in the current turn.
• Inputs required: Assistant answer and the list of role-scoped retrieved document IDs for the turn.
• How to compute it: Extract citation markers via regex, and compute valid_id_rate = valid_ids / all_cited_ids alongside the fraction of answers containing any hallucinated ID.
• Score range & interpretation: 0 to 1 (target 1.0). Treat any fabricated citation ID as a severe failure because it represents a hallucinated source.
• Pitfalls / gotchas: Models sometimes alter punctuation in IDs (doc-12 vs doc_12). Always validate cited IDs against the role-filtered retrieved set so the model cannot leak document IDs from another user's scope (see Access control & RBAC).
• When to use / when not to: Always run this check offline and as a blocking or stripping online guardrail.
• Implemented in: Custom regex check (see code in §4.4).

4.3 Citation coverage (presence)¶

• What it measures: The fraction of factual claims in the answer that carry at least one citation.
• Inputs required: Assistant answer segmented into statements.
• How to compute it: Split the response into statements, filter to citation-worthy factual claims (excluding greetings, hedges, and closing questions), and compute coverage = cited_claims / citation_worthy_claims. Count a claim as cited only if at least one of its citation IDs is valid, so a fabricated ID does not hide an uncited claim.
• Score range & interpretation: 0 to 1. Coverage below 1.0 indicates uncited factual claims drawn from parametric memory or formatting omissions.
• Pitfalls / gotchas: Naive sentence splitters break on abbreviations and bullet lists. Define a clear citation scope rule (such as citing at the end of every sentence) in your prompt and parser.

4.4 Quote verification¶

• What it measures: Whether text enclosed in quotation marks appears verbatim in the cited source document.
• Inputs required: Assistant answer and cited source texts.
• How to compute it: Extract quoted spans, normalize whitespace and Unicode quotes/dashes, and check for an exact substring match in the cited source. If the quote cites only fabricated IDs, mark it unverified; do not search the other sources. Optionally log fuzzy match scores as a diagnostic.
• Score range & interpretation: 0 to 1 (fraction of quotes verified). Treat unverified quotes as severe failures, especially for legal, compliance, or HR policy text.
• When to use / when not to: Always run when your assistant quotes source text, and enforce it as an online guardrail.

import re
import unicodedata
from difflib import SequenceMatcher

CITE = re.compile(r"\[doc:([A-Za-z0-9_\-\.]+)\]")
QUOTE = re.compile(r"[\"“]([^\"”]{12,})[\"”]")          # quotes ≥ 12 chars, straight or curly

def _norm(s: str) -> str:
    s = unicodedata.normalize("NFKC", s)
    s = s.replace("\u2019", "'").replace("\u2018", "'").replace("\u2013", "-").replace("\u2014", "-")
    return re.sub(r"\s+", " ", s).strip().lower()

def split_statements(answer: str) -> list[str]:
    body = re.split(r"\n#*\s*Sources:?\s*\n", answer, flags=re.I)[0]
    parts = re.split(r"(?<=[.!?])\s+(?=[A-Z])|(?<=[.!?\]])\n+|\n\s*[-*]\s+", body)
    return [p.strip() for p in parts if len(p.strip()) > 3]

def citation_checks(answer: str, sources: dict[str, str]) -> dict:
    """sources: {doc_id: text} (the role-filtered documents provided to the model this turn)."""
    statements = split_statements(answer)
    cited_ids = CITE.findall(answer)
    invalid = sorted({i for i in cited_ids if i not in sources})
    claims = [s for s in statements if not s.endswith("?")]    # crude citation-worthiness filter
    uncited = [s for s in claims if not any(i in sources for i in CITE.findall(s))]   # valid IDs only
    quotes = []
    for s in statements:
        for q in QUOTE.findall(s):
            cited = CITE.findall(s)
            ids = [i for i in cited if i in sources] if cited else list(sources)  # fabricated ID -> unverified
            exact = any(_norm(q) in _norm(sources[i]) for i in ids)
            fuzzy = max((SequenceMatcher(None, _norm(q), _norm(sources[i])).find_longest_match().size
                         / max(len(_norm(q)), 1) for i in ids), default=0.0)
            quotes.append({"quote": q, "exact": exact, "longest_match_frac": round(fuzzy, 2)})
    return {
        "valid_id_rate": 1 - len(invalid) / max(len(cited_ids), 1),
        "invalid_ids": invalid,
        "coverage": 1 - len(uncited) / max(len(claims), 1),
        "uncited_claims": uncited,
        "quotes_verified": all(q["exact"] for q in quotes),
        "quotes": quotes,
    }

sources = {"hr-12": "Employees accrue 25 days of paid time off per calendar year.",
           "hr-31": "Unused PTO up to 5 days may be carried over to the next year."}
answer = ('You get 25 days of PTO per year [doc:hr-12]. Up to 5 unused days carry over [doc:hr-31]. '
          'The policy says "unused PTO up to 10 days may be carried over" [doc:hr-31]. '
          'Requests need manager approval [doc:hr-99].')
citation_checks(answer, sources)
# valid_id_rate 0.75 (hr-99 fabricated), coverage 0.75 (the hr-99 claim has no valid citation),
# quotes_verified False (10 vs 5 days; longest_match_frac 0.57)

4.5 Citation faithfulness / support¶

• What it measures: Whether the specific passages cited by a claim actually entail that claim.
• Inputs required: Each claim and the text of the passages it cites. You do not need a reference answer.

• How to compute it: Pair each claim with its concatenated cited passages. Run an NLI model or a binary LLM judge to classify the claim as supported or not supported, then compute citation_support_rate = supported_claims / cited_claims.

  PASSAGES (cited by the claim):
  [hr-31] Unused PTO up to 5 days may be carried over to the next year.
  CLAIM: "Up to 5 unused days carry over."
  Based ONLY on the passages, is the claim fully supported? Paraphrase is fine;
  added specifics (numbers, dates, scope) not in the passages make it unsupported.
  Output JSON: {"reasoning": "...", "supported": true|false}
  

• Score range & interpretation: 0 to 1. This metric correlates directly with whether users find the cited sources trustworthy when they click a link.

• Pitfalls / gotchas: Standard NLI models trained on single sentences struggle with long documents, so chunk long passages or use a validated LLM judge. Instruct the judge to require strict entailment rather than topical similarity. Concatenate all passages cited by a single claim so multi-hop claims are evaluated fairly.
• Implemented in: princeton-nlp/ALCE, custom binary judges, or groundedness metrics in RAGAS, DeepEval, TruLens, and Phoenix (by passing only the cited passages as context).

4.6 Citation recall and precision (ALCE)¶

ALCE formalizes citation support through two complementary metrics (Gao et al., 2023):

For each statement s_i with citation set C_i:

citation_recall(s_i)  = 1  if C_i is non-empty, all IDs in C_i are valid, AND NLI(concat(C_i) ⊨ s_i)
                        0  otherwise
→ "Is the statement fully supported by everything it cites?"

A citation c ∈ C_i is PRECISE (gets precision credit) only if
        citation_recall(s_i) = 1              (the statement is supported by its citations)
    AND c is not IRRELEVANT, where c is IRRELEVANT if
            NOT NLI(c ⊨ s_i)                  (c alone doesn't support s_i)
        AND NLI(concat(C_i \ {c}) ⊨ s_i)      (the others support s_i without c)
→ "Is each citation pulling its weight?"

Per response:
  citation_recall    = (# statements with recall 1) / (# statements)
  citation_precision = (# precise citations) / (# citations, including invalid IDs)
Average both over responses (macro average). F1 of the two is a convenient summary.

Two details matter if you want numbers that you can compare with the paper. First, citations of an unsupported statement never get precision credit. Second, ALCE averages per response, not over all statements or citations in the dataset.

• Worked example: Consider the sentence "Up to 5 unused days carry over [hr-31][hr-12]." Because concat(hr-31, hr-12) entails the claim, citation_recall = 1.0. However, hr-12 alone does not entail the claim, whereas hr-31 alone does. Thus hr-12 is irrelevant, yielding citation_precision = 1/2 = 0.5. If the same sentence cited only [hr-12], recall would be 0 and precision would also be 0, because a citation that does not support its statement gets no credit.
• Pitfalls / gotchas: Scores depend heavily on how you split sentences. Computing precision requires one NLI or judge call per individual citation. Low citation precision usually means the model is padding claims with every retrieved document ID.
• When to use / when not to: Run ALCE recall and precision on offline evaluation sets and sampled online traces.

from typing import Callable

Entails = Callable[[str, str], bool]   # entails(premise, hypothesis): NLI model or validated LLM judge

def alce_response_scores(statements: list[tuple[str, list[str]]], passages: dict[str, str],
                         entails: Entails) -> dict:
    """One response. statements: [(statement_text, [cited_ids])]. Follows ALCE's eval.py logic."""
    n_supported, n_precise, n_cites = 0, 0, 0
    for text, ids in statements:
        n_cites += len(ids)                    # invalid IDs stay in the denominator with 0 credit
        if not ids or any(i not in passages for i in ids):
            continue                           # no citation, or a fabricated ID: recall 0, precision 0
        joint = entails("\n".join(passages[i] for i in ids), text)
        n_supported += joint
        if not joint:
            continue                           # citations of an unsupported statement get no credit
        if len(ids) == 1:
            n_precise += 1
            continue
        for c in ids:                          # irrelevant = fails alone AND the others suffice without it
            rest = "\n".join(passages[i] for i in ids if i != c)
            n_precise += entails(passages[c], text) or not entails(rest, text)
    return {"citation_recall": n_supported / max(len(statements), 1),
            "citation_precision": n_precise / n_cites if n_cites else 0.0}

def alce_scores(responses: list[dict], entails: Entails) -> dict:
    """Macro-average over responses, as ALCE does. responses: [{"statements": [...], "passages": {...}}]."""
    per = [alce_response_scores(r["statements"], r["passages"], entails) for r in responses]
    r = sum(s["citation_recall"] for s in per) / len(per)
    p = sum(s["citation_precision"] for s in per) / len(per)
    return {"citation_recall": r, "citation_precision": p, "citation_f1": 2 * p * r / (p + r) if p + r else 0.0}

# Toy stand-in for an NLI model, only for this example: "entailed" if the premise contains the key fact.
ENTAILS = lambda premise, hyp: "up to 5 days may be carried over" in premise.lower()
passages = {"hr-12": "Employees accrue 25 days of paid time off per calendar year.",
            "hr-31": "Unused PTO up to 5 days may be carried over to the next year."}
alce_response_scores([("Up to 5 unused days carry over.", ["hr-31", "hr-12"])], passages, ENTAILS)
# {'citation_recall': 1.0, 'citation_precision': 0.5}   hr-12 is irrelevant
alce_response_scores([("Up to 5 unused days carry over.", ["hr-12"])], passages, ENTAILS)
# {'citation_recall': 0.0, 'citation_precision': 0.0}   unsupported, so its citation gets no credit

4.7 Limits of automatic attribution checks¶

NLI models and LLM judges make mistakes on citation support. Validate them on your own claim-passage pairs before you gate on them (see LLM-as-a-judge).

• Automatic attribution is hard. On AttributionBench (Li et al., 2024), even a fine-tuned GPT-3.5 reached only about 80% macro-F1 on the binary "is this claim supported by its evidence" task.
• Watch the hard slices. Build test slices for multi-hop claims (the support is split across passages), numeric claims (totals, percentages, unit changes), negation, and claims that rely on tables. Single-sentence NLI models often fail on these, so report support rates per slice.
• Long contexts change the problem. When the model cites spans inside long documents, use sentence-level or span-level citation benchmarks such as LongCite / LongBench-Cite (Zhang et al., 2024) and L-CiteEval (Tang et al., 2024) as references for metric design.

4.8 Citing the right sources (requires gold)¶

When your dataset includes reference contexts (gold document IDs per question, as in SME golden datasets), compute source-level overlap:

cited_source_recall    = |cited_ids ∩ gold_ids| / |gold_ids|
cited_source_precision = |cited_ids ∩ gold_ids| / |cited_ids|

These metrics catch cases where the model cites a tangential document (like an outdated 2021 draft) instead of the authoritative policy. Pair this check with Retrieval metrics so you know whether the retriever failed to fetch the gold document or the generator failed to cite it.

5. Deterministic vs judge: choosing the checker¶

Default stance (heuristic): Always use a deterministic check when one exists; it is free, exact, reproducible, and fast enough to block bad outputs online. Use binary LLM judges or NLI models only for semantic checks, and calibrate them against human labels first.

6. Dataset requirements¶

See Dataset strategy and Synthetic datasets for dataset construction workflows.

7. What to do first¶

• Decompose your system prompt into numbered atomic instructions (§2.1) and rewrite any rule you cannot test.
• Implement deterministic checkers for all verifiable instructions and track prompt-level and instruction-level accuracy.
• Enforce strict tool and output schemas (extra="forbid", explicit enums) and monitor parse and schema validity rates.
• Add deterministic citation checks (ID validity, coverage, and quote verification) and reuse ID and quote checks as online guardrails.
• Build a refusal dataset containing both must-refuse requests and benign look-alikes to measure refusal recall and false-refusal rate.
• Add a binary citation-support judge (or NLI model) and validate it on at least 100 human-labelled claim-passage pairs, including 30–50 unsupported pairs and multi-hop and numeric slices (see LLM-as-a-judge).
• Add ALCE citation precision once citation support is stable so the model does not cite irrelevant documents.
• Plot instruction adherence across turn indices on multi-turn conversations to catch context-window degradation.

Key takeaways¶

• Treat your system prompt as a contract: Decompose it into atomic instructions and assign a test to each one.
• Track both aggregation levels: Use prompt-level accuracy to measure user experience and instruction-level accuracy to prioritize fixes.
• Prefer deterministic checks (word counts, regexes, Pydantic schemas, citation ID lookups, substring quote checks) because they are free, exact, and reusable as online guardrails.
• Use one binary judge per non-verifiable instruction with an explicit n/a option rather than a single 1–10 score.
• Evaluate refusal policies as binary classification by tracking refusal recall and false-refusal rate together on must-refuse and benign look-alike prompts.
• Evaluate citations in layers: Check valid IDs → coverage → entailment support → citation precision. Follow ALCE's exact rules (no precision credit for unsupported statements, per-response averaging) if you compare with published numbers.
• Do not trust automatic attribution blindly: NLI models and judges are weakest on multi-hop and numeric claims, so validate them per slice.
• Validate cited document IDs against the user's role-filtered context so citations never leak unauthorized sources.

Further reading¶

• Zhou et al., 2023, Instruction-Following Evaluation for Large Language Models (IFEval) · code
• Qin et al., 2024, InFoBench: Evaluating Instruction Following Ability in Large Language Models
• Pyatkin et al., 2025, Generalizing Verifiable Instruction Following (IFBench)
• Jiang et al., 2023, FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models
• He et al., 2024, Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following
• OpenAI Model Spec (source of the developer level in the instruction hierarchy)
• Wallace et al., 2024, The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
• Röttger et al., 2023, XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
• Rashkin et al., 2021, Measuring Attribution in Natural Language Generation Models (AIS)
• Bohnet et al., 2022, Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
• Gao et al., 2023, Enabling Large Language Models to Generate Text with Citations (ALCE) · code
• Li et al., 2024, AttributionBench: How Hard is Automatic Attribution Evaluation?
• Zhang et al., 2024, LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA
• Tang et al., 2024, L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding?
• Pydantic · jsonschema (Python)
• In this repo:
  • LLM-as-a-judge
  • Output guardrails
  • Metric cheatsheet