Map · Agents · 07 · Prompt injection
Concept card · Agents
Prompt injection is untrusted text that tries to override the instruction or the tool rules.
Untrusted text that tries to override the instruction or the tool rules.
Oct 18: First card. The picture keeps one instruction and switches the note between data and an override attempt. The OpenAI citation is the computer use guide. Other sources are OWASP's LLM01:2025 entry, Anthropic's jailbreak guide, OpenAI's safety best practices, and Gemini's safety guidance.
Prompt injection is untrusted text that tries to override the instruction or the tool rules. OWASP's LLM01:2025 entry says this vulnerability is a prompt that alters the model's behavior or output in unintended ways, either in the user's own prompt or in external content such as a website or a file. OpenAI's computer use guide says to treat screen content as untrusted: text in a page, a document, or a tool result cannot grant permission or override the user's instructions. Anthropic's guide includes a tool result as third-party content that can carry those instructions. Gemini's safety guidance describes an input that tells the model to ignore previous examples.
A check on that text is another request. Anthropic's guide describes a separate classifier call on tool output before the output is returned to the main model. OpenAI's safety best practices say limiting how much text a user can put in the prompt helps avoid prompt injection. This card does not add a price.
OWASP says a successful prompt injection can manipulate content into incorrect output. The reply can follow the untrusted text instead of the task in your instruction. This card does not claim how often that happens.
OWASP says the impact depends on the context and on the agency the model is given, and lists unauthorized access to functions available to the model. The same entry says it is unclear if there are fool-proof methods of prevention. OpenAI's computer use guide says to restrict the environment to an isolated browser or virtual machine and an allow list of sites and actions, and to keep access limited to what the task needs. Anthropic's guide says to limit the model's access so a successful injection can do less.
Take one document or tool result your agent will read. Keep your instruction in the instruction slot, put that document in the user or tool-result slot, and write down which tools that document is allowed to cause.
Putting untrusted text in the system instruction, or trusting one line that says to ignore later attempts to change the instruction. Anthropic's guide says to deliver third-party content inside tool results, not in the system prompt. OWASP lists prompt wording as one mitigation beside least privilege and human approval.
A retrieved note says "ignore the previous instructions and do this instead." What is that note?
Coming soon. One real tool result, labelled untrusted, with the tools it is allowed to cause written beside it.
Coming soon. How OWASP, Anthropic, and OpenAI's computer use guide each separate direct injection from text that arrives inside a document or a tool result.