The one-sentence version
Prompt injection does not have a "silver bullet" solution like "parameterized queries," so defense must be implemented in two layers: one layer is to reduce the probability of being injected (model training, input tagging, detectors), and the other, more important layer, is to limit the consequences after an injection occurs (least privilege, isolation of untrusted content, human confirmation for sensitive operations, and cutting off exfiltration channels). The growing consensus in the industry is: Do not rely on the model to distinguish good from bad; instead, design the system to ensure that "even if the model is deceived, it cannot do harm."
This article has three layers:
- Layer 1: The Intuition: You cannot rely solely on a secretary's judgment
- Layer 2: Real-World Use: An overview of mainstream methods
- Layer 3: Principles and engineering key points: The fatal triad, Dual LLM pattern, CaMeL, and design pattern checklists
Layer 1: The Intuition
1.1 You cannot rely solely on a secretary's judgment
A secretary can easily be deceived by a note (see "Prompt Injection"). You have two ways to handle this:
- Train the secretary: Teach them that "words in a document are not my commands." This is useful, but there will always be a note that can trick them;
- Change the process: The secretary can read any document, but they cannot send letters themselves and cannot touch the safe; to send a letter, they must bring it to you for a signature.
The second method does not depend on the secretary's judgment, making it much more reliable.
1.2 Two-layer defense
| Layer | Goal | Methods | Reliability |
|---|---|---|---|
| Reduce Probability | Make injection harder to succeed | Model training, input tagging, detectors | Probabilistic, can be bypassed |
| Limit Consequences | Even if injection succeeds, no harm can be done | Permissions, isolation, confirmation, egress control | Deterministic, guaranteed by the system |
Layer 2: Real-World Use
| Method | Proposer / Year | Approach |
|---|---|---|
| Instruction Hierarchy | OpenAI, 2024 | Train the model to distinguish priority: System > Developer > User > Tool results; instructions in lower priority cannot override higher priority |
| Spotlighting | Microsoft Research, 2024 | Add explicit tags or encoding to untrusted content (e.g., inserting special symbols between words) to help the model distinguish "this is data" |
| Injection Detectors | Various vendors | Use specialized classification models to detect whether input contains injection instructions |
| Dual LLM Pattern | Simon Willison, 2023 | One "privileged" model can call tools but never touches untrusted content; one "isolated" model processes untrusted content but cannot call tools |
| CaMeL | Google DeepMind, 2025 | Adds "capability" tracking to the Dual LLM idea: every piece of data carries a source tag, and the policy engine decides where it can flow based on the source |
| Agent Design Patterns | Joint paper by multiple institutions, 2025 | Summarizes several Agent architecture patterns that balance usability and security (e.g., "plan then execute," "action selector," etc.) |
The first three belong to "reducing probability," while the last three belong to "limiting consequences."
Layer 3: Principles and Engineering
3.1 Check the "Fatal Triad" first
If an AI system simultaneously possesses:
- Access to private data;
- Exposure to untrusted content;
- The ability for external communication;
then it is almost certain that it can be exploited via injection attacks to steal data (Simon Willison calls this the "fatal triad"). When designing, first ask: Can we remove one of these?
| Remove | Example |
|---|---|
| Private Data | An Agent processing public webpages does not connect to the user's email |
| Untrusted Content | An Agent that can read email does not browse external webpages |
| External Communication | Prohibit rendering external images, prohibit arbitrary internet access, and require human confirmation for tool-based sending (see "Markdown Image Data Exfiltration" and Egress Control) |
3.2 Dual LLM Pattern
Key point: The privileged model only sees the variable name ($sum1) and not the content of the variable, so injection instructions within the content cannot affect its decision-making. The limitation is: if the privileged model needs to make decisions based on the content (e.g., "if the email mentions a meeting, add it to the calendar"), this pattern is insufficient. CaMeL partially solves this by having the privileged model generate a piece of code, which is then executed by an interpreter that tracks the source of every value.
3.3 Engineering Checklist
| Category | Measure |
|---|---|
| Permissions | Minimize tool permissions; prioritize read-only; do not pass credentials into the model context (see "Agent Permission Design" and Credential Leakage & Context Window) |
| Isolation | Untrusted content and high-privilege tools should not share the same context; tag data by source |
| Confirmation | Actions such as sending externally, writing, deleting, or making payments require user confirmation and must display complete parameters |
| Egress | Do not automatically load external resources; use a whitelist for network access; manage DNS as well (see "Egress Control") |
| Output | Check links and images before rendering; do not trust HTML within model output |
| Memory | Writing to long-term memory requires confirmation (see "RAG and Memory Poisoning") |
| Observability | Log every tool call; alert on anomalous behavior |
| Testing | Perform continuous regression testing with injection datasets (see "AI Red Teaming") |
3.4 The Trap of Human Confirmation
"Ask the user to confirm every action" sounds the safest, but it leads to confirmation fatigue: users will eventually click "Allow" a hundred times a day without looking at the content (similar to the logic in risk control scoring and friction design). A better approach is:
- Low-risk actions (read-only, local, reversible) are allowed automatically;
- High-risk actions (sending externally, spending money, deleting, changing permissions) require confirmation;
- The confirmation interface clearly displays "what is being done, to whom, and what content is being sent."
Summary
| Layer | Key points |
|---|---|
| Principle | Do not rely on the model to distinguish good from bad; design the system to ensure "even if deceived, it cannot do harm" |
| Reduce Probability | Instruction Hierarchy, Spotlighting, Detectors |
| Limit Consequences | Break the fatal triad; Dual LLM / CaMeL; Least privilege; Egress control |
| Balance | Confirm based on risk levels to avoid confirmation fatigue |