What is prompt injection?
Prompt injection is an attack where text a model reads overrides the developer's instructions.
Prompt injection is an attack in which text that a language model reads changes what the model does, against the wishes of the developer who built the application. OWASP’s 2026 entry says it happens when input to an LLM, whether direct user input, retrieved content, tool output, image, audio or video content, intermediate reasoning or persistent memory, alters the model’s behavior in ways the developer did not intend. It is LLM01:2026 in the OWASP GenAI LLM Top 10 (published August 4, 2026), and the entry states that “no reliable prevention mechanism exists today.”
Where the name comes from
Simon Willison proposed the name on September 12, 2022, writing that “the obvious parallel here is SQL injection.” Both attacks work when untrusted input is joined to trusted instructions in one string. The difference is that SQL has parameterized queries to keep code and data apart, and a model reads instructions and data in the same stream of text.
Direct and indirect prompt injection
OWASP’s 2026 entry defines two types, by how the text reaches the model.
| Type | How it arrives | Source of the instruction |
|---|---|---|
| Direct | A user supplies input that alters behavior, on purpose (a jailbreak) or by accident (pasting content with conflicting instructions) | The person typing |
| Indirect | The model reads external content containing hidden instructions the user never wrote or saw | Content an attacker may control or have reached |
OWASP lists multimodal payloads hidden in images, audio or video as a common example, along with poisoned memory or retrieval data that affects later sessions. For indirect injection it grades the source by trust: untrusted (the public web, unknown email), semi-trusted (public bug trackers, package READMEs) or trusted (internal repositories that an attacker reached through an upstream channel).
Indirect injection matters most for agents, because an agent reads email, web pages, documents and tool results that no one on your team wrote. NIST’s agent security team calls it agent hijacking: an attacker hides instructions in data the agent reads, such as emails, websites or code repositories, to make it leak data or run malicious code.
Prompt injection is not jailbreaking
Willison draws the line at what is attacked. Prompt injection targets applications by “concatenating untrusted user input with a trusted prompt” the developer wrote. Jailbreaking targets “safety filters built into the LLMs themselves.” He adds that jailbreaking mostly risks embarrassing output, while prompt injection threatens any application with confidential data or privileged tools.
When it becomes serious: the lethal trifecta
Willison’s lethal trifecta (June 16, 2025) names three capabilities that, together in one agent, let an attacker steal data:
- Access to private data.
- Exposure to untrusted content.
- The ability to communicate externally.
He writes that models cannot reliably tell instructions apart by where they came from, and that “we still don’t know how to 100% reliably prevent this from happening.” For users, he says the dependable defense is to avoid combining all three. Check each agent you build or install against this list before you look at filters. For how tools get attached to agents, see the Model Context Protocol.
How hard is it to defend?
Published results show the risk falling but not vanishing.
| Source | Finding |
|---|---|
| Anthropic (Nov 24, 2025) | Against an adaptive attacker with 100 attempts per environment, Claude Opus 4.5 in browser use was attacked successfully 8.0% of the time with the model alone and 1.4% with improved safeguards. Anthropic says a 1% rate “still represents meaningful risk.” |
| NIST CAISI (Mar 23, 2026) | In a public red-teaming contest against 13 frontier models, more than 400 participants made over 250,000 attack attempts, and at least one attack succeeded against every model. Some attacks transferred across models. |
These figures use different attackers, environments and models, so do not compare them directly. Rerun any rate on your own agent before you trust it.
Defenses that lower the risk
No single control is enough. The providers and OWASP list overlapping layers. OWASP’s 2026 entry says the goal is to limit damage after an injection succeeds.
| Layer | What it does | Source |
|---|---|---|
| Train the model | Reinforcement learning on simulated injected content teaches the model to refuse the instructions | Anthropic |
| Classify untrusted content | A model scans content before the main model acts on it | Anthropic, Google |
| Constrain output | Enums and fixed schemas remove freeform channels attackers could use to smuggle instructions or data | OpenAI |
| Separate roles | Route user and external input through user messages, not developer messages, which carry higher authority | OpenAI |
| Least privilege in code | Keep credentials and state-changing actions in application code, with least-privilege access and a deterministic policy check when the action runs | OWASP 2026 |
| Human approval | Require confirmation for privileged, irreversible or externally visible actions | OWASP 2026, OpenAI, Google |
| Sanitize output | Do not render external image URLs and redact suspicious links, which closes a data exfiltration path | |
| Test adversarially | Test against adaptive attackers who know your defenses, not only static benchmarks | OWASP 2026 |
| Limit capabilities | Meta AI’s Rule of Two: an agent with untrusted input, sensitive data and the ability to change state or communicate externally needs per-action human approval | OWASP 2026 |
| Validate output, filter input | Validate outputs against a strict schema in code, filter every input modality, strip invisible Unicode characters | OWASP 2026 |
Google frames its layers as raising the difficulty, expense and complexity an attacker faces, not as a cure. OpenAI says its guardrail nodes are not foolproof but are an effective first wave of protection. Willison warns that guardrail products claiming to catch 95% of attacks fall short of the standard expected in web security.
Prompt wording alone is not enough. OWASP’s 2026 entry lists defining the model’s role in the system prompt as one mitigation but calls it “a partial control only” (OWASP 2026), and Willison asks of prompts that tell a model to ignore injected instructions, “how confident can you be that your protection will work every time?”
Design the agent to survive a successful injection
A paper by Beurer-Kellner and colleagues (June 2025) proposes design patterns for agents that resist prompt injection and discusses their utility and security trade-offs. Willison quotes its principle: once an agent has ingested untrusted input, it must be constrained so that it cannot trigger consequential actions. In practice, that means deciding which tools an agent can reach after it reads outside content, not hoping a filter catches every attack.
For the runtime checks that sit around a model, see AI guardrails. For the agent loop that makes injection consequential, see what AI agents are.
Spotting injection attempts in HoneyHive
Prevention happens in your code and architecture. HoneyHive records what happened so you can find attempts and check your defenses. Its custom metrics guide says guardrails such as safety checks are ideal to compute client-side with enrich_span(metrics=...). Log a metric such as injection_flagged as 0 or 1 on the span that read the untrusted content, then set an alert on its average. Checks that only score can run as online evaluations, which run asynchronously after ingestion. See agent observability for tracing tool calls across an agent run.
Related: AI red teaming.
FAQ
Is prompt injection the same as jailbreaking?
No. Simon Willison separates them by what is attacked: prompt injection targets applications that join untrusted input to a trusted prompt, while jailbreaking targets the safety filters built into the model itself. His test is that "if there's no concatenation of trusted and untrusted strings, it's not prompt injection."
Can prompt injection be fully prevented?
Not today. OpenAI's agent safety guide says structured outputs and isolation "greatly reduce, but don't fully remove" the risk, and Anthropic says no browser agent is immune. Design the system so a successful injection cannot do much damage.
What is indirect prompt injection?
It is prompt injection where the instructions arrive through content the model reads, not through the user's own message. OWASP's 2026 entry describes the model reading external content with hidden instructions that the user never wrote or saw, such as web pages, emails or documents.