# Guardrails > A guardrail is an automatic check at a fixed point in your agent's run. When the check fails, the run gets a fixed reply, a retry, a rerouted handoff, or a stop. Check what goes into and comes out of your agent, and stop or reroute a run when a check fails. Track: Agents Card: https://buildsparkai.com/guardrails Plain text: https://buildsparkai.com/guardrails.md Last checked: 2026-10-21 ## What it is A guardrail is an automatic check at a fixed point in your agent's run: on the input, around a tool call, on the reply, or on the run as a whole. When the check fails, it changes what happens next. The run gets a fixed reply, a retry, a rerouted handoff, or a stop. ## Why it matters **Cost.** A check before the main AI model can stop an off-task request before the expensive call runs. **Quality.** Checking a tool's values catches the wrong account before a ticket lands on it. **Risk.** [OWASP, LLM10:2026 Improper Output Handling](https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/) says to treat the model like any other user and validate what it sends on. Encode its output for the page it lands on. ## Try today Pick one tool your agent calls that changes something. Write one check for its arguments, put it right before the call, and write what happens when it fails, and when the check itself is down. ## Common mistake Checking only the first message. In a run with several steps or agents, the input check runs once. Text that arrives later from tools doesn't meet it. Put a check next to the tool. ## Self-check A check makes sure create_ticket gets the signed-in customer's account id. Where does it go? Right before create_ticket runs ## Understand it: where a check runs, and what it can't catch 6 min, optional A guardrail is a decision you wrote down before the request arrived. The main model doesn't get to waive it. ### Where each check runs [OpenAI, Guardrails and human review](https://developers.openai.com/api/docs/guides/agents/guardrails-approvals) places input guardrails on the first agent and output guardrails on the last. Tool guardrails run on the tools they're attached to. [OpenAI Agents SDK, Guardrails](https://openai.github.io/openai-agents-python/guardrails/) says the same about a chain: an input check covers the start, an output check covers the end, and a tool check runs on every guarded call. A parallel input check is the SDK's default. It can be faster, and the main agent may already have run tools before the check trips. A blocking check, `run_in_parallel` set false, stops those tools first. Block when a tool can change something. A tripwire then stops the run, reroutes to the person named on [Permissions](/permissions-hitl#handoff), or returns a fixed reply. Don't let the main model decide whether its own check matters. ### Checks in the server [Model Context Protocol, Tools](https://modelcontextprotocol.io/specification/2026-07-28/server/tools) says servers must validate inputs, enforce access controls, rate-limit, and sanitize outputs. Clients should validate results before passing them to the language model, set timeouts, and log tool usage. A check that lives only in the prompt is the same miss as on Permissions: the model can still propose the call. ### Escape what you display Model output can carry markup or code. [OWASP, LLM10:2026 Improper Output Handling](https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/) says to encode output for the page it lands on and to use parameterized queries. [Google, Safety and Security for AI Agents](https://adk.dev/safety/) says to escape model output in the UI, and to run in-tool checks and before-tool callbacks. A callback can also have a cheap, fast model screen inputs and outputs. ### A second model, and the limits that stop a run [Anthropic, Building effective agents (published Dec 19, 2024)](https://www.anthropic.com/research/building-effective-agents) says a separate screening call tends to perform better than one call that both does the task and checks it. Treat that verdict as one signal, the way [Safety](/safety) treats a moderation flag, and test it the way you test any other check. [Test your checks: Evaluation basics.](/evaluation-basics) Cap steps, repeated failures, calls per minute, and how long the run may wait. [Your stop rules from Agent loop](/agent-loop#u-stops) are the done line, the turn limit, and the failure limit. A guardrail adds the rate and the timeout, and it says what happens when a check is down. [NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) asks, in MANAGE 2.4, for an assigned way to supersede, disengage, or deactivate a system whose outcomes don't match the intended use. MS-4.2-004 says to record when a person or another system overrides a decision. ### What guardrails can't do They catch what you thought to check. A missed check can still reach a tool that is on the list. Keep [Permissions](/permissions-hitl) so an act tool isn't available unless this task needs it, and so a person still approves the action. [What your product won't do at all: Safety.](/safety) See the five questions on the [support assistant](/practice/support-bot/part-2#eval). That step checks whether an answer matches the docs. ### Engineer notes · checks you can copy optional, for builders shipping act tools For when you wire the checks yourself: where each one runs, what blocks, and how to count hits and misses. The rules and the queue are yours. 1. **Input:** before the main AI model. Parallel only while nothing can act. 2. **Tool in:** blocking, next to each tool. No rule, no call. 3. **Approval:** act tools wait in your [Permissions](/permissions-hitl#handoff) queue after their check passes. No answer means no. 4. **Tool out:** labelled untrusted before the model reads it ([Prompt injection](/prompt-injection)). 5. **Output:** redact, cite and escape before the reply leaves. A check that errors or times out holds the run for a person. That's failing closed. A sketch, not a library. Call it where the [Agent loop notes](/agent-loop#u-engineer) run a tool. ```js // Sketch. RULES, tools and queue are yours. export async function guardedCall(call, user, tools, key) { const tool = tools[call.tool]; const v = await checkIn(call, user); // blocking if (v.verdict === "block") return { ok: false, error: v.why }; // loop fixes if (v.verdict === "hold") return queue.hold(call, v.why); // pause, hand off if (tool.acts && !(await queue.ask(call, user))) return { ok: false, error: "not approved" }; return checkOut(await tool.run(call.args, { key })); } export async function checkIn(call, user) { const rule = RULES[call.tool]; if (!rule) return { verdict: "block", why: "no rule" }; try { const why = await within(rule(call.args, user), 300); return why ? { verdict: "block", why } : { verdict: "pass" }; } catch { return { verdict: "hold", why: "check down" }; } } function checkOut(res) { if (!res.ok) return res; // errors stay errors const text = JSON.stringify(res.data ?? "").toLowerCase(); const flags = FLAGS.filter((f) => text.includes(f)); return { ...res, untrusted: true, flags }; // data } const RULES = { search_docs: async () => null, // reads: still a rule create_ticket: async (args, user) => args.account === user.account ? null : "account doesn't match", }; const FLAGS = ["ignore previous", "system prompt"]; const within = (p, ms) => Promise.race([p, new Promise((_, no) => setTimeout(no, ms, "timeout"))]); ``` - **Blocking:** `checkIn()` returns before the tool runs. - **Fail closed:** no rule blocks, and a slow or broken check holds. A hold stops your loop. It isn't a failure to retry. - **A block is an error:** the loop sees `ok: false` and can fix the call. - **Flags are labels:** flagged text stays data, never orders ([Failure modes](/failure-modes)). A phrase list misses rewordings. **Trade-offs.** A stricter rule misses less and blocks more good calls. A false block stalls a task; a miss lets a bad act run. Lean strict on act tools. Each blocking check adds its time to every call: keep rules fast, and time-limit any check that calls a model. Label cases by hand: good calls that should pass, bad ones that should stop. Per tool, count hits, misses, false blocks and the slowest time. Add each later miss as a case. ```js const by = {}; for (const c of cases) { const t0 = Date.now(); const v = await checkIn(c.call, c.user); const stop = v.verdict !== "pass"; const r = (by[c.call.tool] ??= { hit: 0, miss: 0, falseBlock: 0, pass: 0, maxMs: 0 }); if (c.bad) stop ? r.hit++ : r.miss++; else stop ? r.falseBlock++ : r.pass++; r.maxMs = Math.max(r.maxMs, Date.now() - t0); } ``` [Agent evaluation covers graded sets and repeat runs →](/agent-evaluation) Your account check times out on a `create_ticket` call. What should happen, and how does your eval count it? Answer: The run holds for a person, and no ticket is filed. The eval counts a stop: a hit if the case was bad, a false block if it was good. ## Practice and deep Example: made up to show the idea. A six-event trace for the same support assistant. The first two and the last are worked for you. Place the check for the three events in between. - e1, the user message. Input check: passes. Why: it asks for a ticket, not a password. - e2, a search_docs result that includes the pasted note. After-tool check: labelled untrusted, kept as data. Why: the note is text the model reads, not a rule your app follows. - e6, the output checker times out. The reply is held and handed to the support person. Why: a check that can't run pauses, rather than passing. Self-check e3: create_ticket is called with account 4471. The signed-in account is 2208. Where does the check go, and what happens? A · On the output: block the reply B · On the input: reject the message C · Before the tool: send it back ## Sources - [Anthropic, Building effective agents (published Dec 19, 2024)](https://www.anthropic.com/research/building-effective-agents) - [Google, Safety and Security for AI Agents](https://adk.dev/safety/) - [Model Context Protocol, Tools](https://modelcontextprotocol.io/specification/2026-07-28/server/tools) - [NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) - [OpenAI, Guardrails and human review](https://developers.openai.com/api/docs/guides/agents/guardrails-approvals) - [OpenAI Agents SDK, Guardrails](https://openai.github.io/openai-agents-python/guardrails/) - [OWASP, OWASP GenAI LLM Top 10 2026: LLM10:2026 Improper Output Handling (LLM means large language model)](https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/) Anthropic's guide was published Dec 19, 2024. The Tools specification is dated 2026-07-28. NIST's profile was published July 26, 2024.