How to Detect Prompt Injection: Four Approaches Ranked
Input heuristics, classifier APIs, hidden-state probes, and output monitoring: how to detect prompt injection in production LLM apps, with tradeoffs.
How to detect prompt injection is not a solved problem, and anyone who tells you otherwise is selling you a single-layer guardrail with a bypass rate they haven’t measured. The research published through 2025 is clear: no individual technique covers the full attack surface. What works is a layered stack, and understanding each layer’s failure mode is what lets you stack them intelligently.
OWASP’s LLM01:2025 entry defines two distinct attack classes you need to cover. Direct injection arrives in the user turn — the attacker types it. Indirect injection arrives in external data the model retrieves: a RAG chunk, an email it’s summarizing, a web page a browser agent fetched. Most detection products focus on direct injection. Indirect injection is where production incidents actually happen.
Approach 1: Input Heuristics and Signature Scanning
The simplest layer is a regex or keyword filter that flags known injection strings: “ignore previous instructions,” “you are now,” “disregard your system prompt,” and their immediate variants. You can extend this with perplexity scoring — injected payloads often contain anomalously high-entropy token sequences relative to normal user input.
What it catches: naive, copy-paste attacks. Script-kiddie injection that’s unchanged from public PoC repos.
What it misses: anything paraphrased, base64-encoded, written in another language, or split across multiple turns. Adversarial research consistently shows these filters are defeated by trivial obfuscation. A Unicode homoglyph substitution or a single synonym swap breaks most regex rules.
Use this layer because it’s nearly free. Don’t rely on it as your primary signal.
Approach 2: Classifier-Based Detection
Classifier-based detection trains a model to distinguish injected from benign inputs. The most common production approach is a fine-tuned encoder: Meta’s PromptGuard fine-tunes a DeBERTa model for binary injection/benign classification. Lakera Guard and LLM Guard both offer classifier-API endpoints that wrap similar architectures behind an HTTP call.
Research on embedding-based classifiers shows that traditional ML models — LSTM, random forest, naive Bayes — trained on the HackAPrompt corpus achieve meaningful detection rates. More sophisticated approaches encode the input into an embedding space and measure cosine distance from clusters of known injection payloads.
What it catches: injection patterns that appear in training data, including many variants the regex layer misses.
What it misses: out-of-distribution payloads. A 2025 paper on bypassing LLM guardrails demonstrates that character injection and adversarial machine learning evasion techniques can bypass detection while keeping the attack payload functional. Classifiers trained on yesterday’s attack corpus don’t generalize to novel payload structure.
False positives are a real operational cost here. Legitimate inputs that look like instructions — software documentation, code reviews, technical support queries — trip classifiers trained on injection-heavy corpora. AlignSentinel addresses this with a three-class system (benign instruction / malicious injection / normal input) using attention map features, but it requires access to model internals.
Approach 3: Hidden-State and Attention Probes
This is the highest-reliability approach available, and it requires the most privileged access: you need to hook into the intermediate layers of the model itself.
PIShield extracts hidden states from transformer layers and feeds them into a lightweight probe classifier without requiring labeled injection data. AttentionTracker uses attention pattern analysis at inference time to flag inputs that redirect the model’s attention toward injected instruction tokens. Both approaches work on the principle that injection attempts create detectable signatures in how the model internally processes text, independent of the surface-level token content.
What it catches: injection attempts that survive input-layer classifiers, because the detection signal comes from model internals rather than input text features.
What it misses: attacks sophisticated enough to manipulate attention patterns in ways that mimic benign processing — an active area of adversarial research. It also misses anything before the model processes it; it’s a detection mechanism, not a prevention mechanism.
Deployment constraint: you need access to model internals. This works if you’re running your own inference (vLLM, llama.cpp, a fine-tuned endpoint). It doesn’t work if you’re calling GPT-4o or Claude via API without layer access.
For teams without model internals, the practical equivalent is an LLM-as-judge layer: pass the input to a secondary model with a detection instruction. This is expensive and introduces a meta-injection surface (inject into the detector), but it’s the closest approximation available with hosted APIs.
Approach 4: Output and Behavior Monitoring
Input-side detection is blind to indirect injection that arrived in retrieved context. The model already processed the poisoned RAG chunk or web page before your filter ran. Output monitoring is the backstop.
Watch what the model does, not just what it receives. Specific signals worth monitoring:
- Schema violations: the model was supposed to return JSON with specific fields; it returned free text or unexpected keys
- Tool call anomalies: the model called a tool it wasn’t instructed to use, or passed arguments that don’t match the task (e.g., exfiltrating user data to a URL endpoint)
- Persona breaks: systematic detection of the model asserting a different identity or refusing system-prompt constraints
- Unexpected external references: the model’s output cites URLs, names, or instructions that don’t appear in the legitimate context
This layer is genuinely effective against indirect injection because it observes the effect rather than the cause. The OWASP LLM01 guidance specifically calls out monitoring tool calls with human approval gates for high-risk actions as a primary mitigation.
More on the defensive tooling side: guardml.io covers guardrail frameworks and aisec.blog tracks production prompt injection cases with real-world incident context.
What This Stack Doesn’t Fully Solve
Indirect injection remains the hardest problem. When the attack payload arrives inside a document the model was legitimately instructed to process, the semantic boundary between “content to analyze” and “instruction to follow” is exactly what the model was trained to blur. Spotlighting — using special delimiter tokens to separate instruction zones from data zones and training the model to flag cross-boundary commands — helps but doesn’t eliminate the problem.
The 2025 ACM Workshop paper on detection limits argues that any detection-based defense has a theoretical accuracy ceiling, and adversaries with access to the detection system can calibrate payloads to sit below the detection threshold. Defense-in-depth is not a clever suggestion; it’s the only honest recommendation given the constraint.
Practical Deployment Order
- Regex + perplexity filter on all input turns — cheap, catches volume attacks
- Classifier API (Lakera Guard, LLM Guard, or PromptGuard locally) on user-facing inputs
- External content treated as untrusted — run classifier on RAG chunks and fetched content before injection into context, not just on the user turn
- Output schema validation and tool-call monitoring — gate high-risk tool calls with an explicit approval check
- Hidden-state probes if you control your inference stack — highest signal, lowest false-positive rate
Red-team your detection layer separately from your application. Bypass research is public and active; assuming your guardrail held because you haven’t seen an incident is exactly the failure mode that shows up in breach reports six months later.
Two follow-ups make that concrete. Why static jailbreak filters fail covers the published results from attacking twelve deployed defenses adaptively, including which detection families held and which did not. The catalog of documented jailbreak patterns lists the specific attack classes each detection approach is blind to, which is the shortest way to work out how many independent layers a given deployment needs.
Sources
- OWASP LLM01:2025 Prompt Injection
- PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Features (arXiv 2510.14005)
- Detecting Prompt Injection Attacks Against Applications Using Classifiers (arXiv 2512.12583)
- Bypassing Prompt Injection and Jailbreak Detection in LLM Guardrails (arXiv 2504.11168)
AI Attacks — in your inbox
Practitioner-grade AI red team techniques and tooling — delivered when there's something worth your inbox.
No spam. Unsubscribe anytime.
Related
How Indirect Prompt Injection Works
How indirect prompt injection works: attacker instructions hidden in web pages, emails and RAG documents, the attack surface, and what defenders can do.
LLM Jailbreak Examples: 10 Documented Patterns
Ten LLM jailbreak examples drawn from published research, with the reported success rates, the mechanism behind each, and the signals that detect them.
Prompt Injection vs Jailbreak: How They Differ and Why It Matters
Prompt injection targets your application architecture; jailbreaking targets the model's safety alignment. Confusing them defends the wrong layer.