Prompt injection has become one of the defining security problems of modern AI applications. As large language models move beyond simple chat interfaces and gain access to internal documents, APIs, business systems and autonomous tools, the consequences of manipulating model behavior become substantially more serious.

A prompt injection attack attempts to influence an AI system through information the model interprets as instructions. The malicious input may come directly from a user, but it can also arrive indirectly through documents, webpages, emails, retrieval systems or other content processed by an AI application.

OWASP currently classifies Prompt Injection as LLM01:2025, defining the vulnerability as a situation in which user prompts alter an LLM’s behavior or output in unintended ways. OWASP also emphasizes that malicious instructions do not necessarily need to be clearly visible to a human as long as the model can parse them.

The important security lesson is that prompt injection is not simply a problem of users typing clever instructions into a chatbot. It is a trust-boundary problem created when applications allow natural-language content to influence models that have access to sensitive information or privileged capabilities.

Understanding that difference is essential for anyone building AI-powered software in 2026.

What Is Prompt Injection?

Prompt injection occurs when content supplied to a large language model changes how the model interprets its intended instructions.

An LLM application normally receives several sources of context. These may include developer instructions, system configuration, user input, retrieved documents, tool responses and conversation history. From the model’s perspective, much of this information exists inside the same reasoning environment.

That creates a fundamental security challenge.

Traditional software can usually distinguish code from data through explicit technical structures. Large language models operate primarily on language and context. Instructions and data can therefore become difficult to separate reliably when both are expressed as natural language.

OWASP notes that prompt injection can manipulate model behavior directly or indirectly and that these attacks may result in consequences such as unauthorized information disclosure, manipulated decision-making or unintended interaction with connected capabilities.

A simple chatbot with no access to sensitive systems may suffer little more than an inappropriate response.

An AI agent with access to customer information, internal APIs and administrative tools creates a very different risk.

The injection technique may be similar.

The security impact depends on the architecture surrounding the model.

Why LLM Applications Are Vulnerable to Instruction Manipulation

Large language models are designed to interpret instructions rather than enforce security policy.

That distinction is central to understanding prompt injection.

A developer may give an AI application instructions describing how it should behave. The application may tell the model which information it should disclose, what tone it should use or which actions it should avoid.

Those instructions influence model behavior, but they are not equivalent to deterministic authorization controls.

A sufficiently unusual or adversarial context may cause the model to interpret competing instructions differently from what the application developer intended. This is especially difficult when the system intentionally allows users to provide open-ended natural-language input.

NIST includes prompt injection within the broader field of adversarial machine learning and organizes AI attacks according to attacker goals, capabilities, knowledge and the stage of the AI lifecycle being targeted.

This broader perspective is useful because prompt injection is not only about the words sent to a model. The attack depends on what the surrounding system permits the manipulated model to influence.

Direct vs Indirect Prompt Injection

Two broad prompt-injection scenarios are particularly important for AI application security: direct prompt injection and indirect prompt injection.

The difference is determined primarily by where the adversarial instruction enters the AI workflow.

Direct User-Controlled Instructions

Direct prompt injection occurs when a person interacting with an AI application intentionally provides content designed to interfere with the model’s intended behavior.

The attacker controls the input channel directly.

For example, an application may expose an AI assistant through a web interface. A user can then attempt to influence the assistant into ignoring intended restrictions, disclosing protected context or using connected functionality differently from the application’s intended workflow.

The security significance of that manipulation depends heavily on what the assistant can access.

If the model has no sensitive context and no privileged tools, the effect may remain limited to unusual output.

If the same model can query internal records or invoke backend functions, a manipulated instruction may become the first stage of a more serious attack chain.

For that reason, security teams should avoid evaluating direct prompt injection solely by asking whether a user can make the model produce prohibited text.

The better question is:

Can manipulated instructions cross a security boundary?

Malicious Instructions Hidden in External Content

Indirect prompt injection creates a more difficult security problem because the attacker may never interact with the AI application directly.

Instead, adversarial instructions are placed inside content that the AI later consumes.

That content might originate from a webpage, email, uploaded document, support ticket, knowledge base, external dataset or retrieval-augmented generation system.

NIST’s 2026 work discussing agentic AI security specifically highlights the problem of attackers embedding new objectives inside natural-language artifacts such as PDFs, emails and RAG content.

Imagine an AI assistant that reads external webpages to summarize research.

The user believes the webpage is data.

The AI also sees natural language.

If hostile content inside that page can influence the model’s instructions, information that was supposed to be processed may instead become information that controls processing.

This is why indirect prompt injection is particularly important for autonomous agents.

An attacker can potentially influence an AI system simply by controlling something the agent is expected to read.

Prompt Injection Is Really a Trust-Boundary Problem

Treating prompt injection as a prompt-engineering issue leads to weak defenses.

The deeper problem is that AI applications frequently combine trusted instructions and untrusted content inside the same reasoning process.

Consider a simplified AI workflow:

A user asks an assistant to analyze a document.

The application retrieves the document.

The document is placed in the model context.

The model interprets both the user’s request and the document contents.

If the document contains adversarial instructions, the model must determine which text represents data and which text represents behavior it should follow.

That distinction is not equivalent to an operating system separating executable code from a text file.

The safest application therefore assumes that untrusted content may influence the model.

Security controls should be designed around that assumption.

Anthropic’s research on prompt-injection defenses for browser-using agents describes prompt injection as one of the major security challenges for agents interacting with external content.

The objective is therefore not to create a magical prompt that can never be manipulated.

The objective is to ensure that manipulation cannot easily produce dangerous consequences.

How Prompt Injection Becomes a Real Security Problem

Not every successful manipulation represents the same level of risk.

An AI system producing an unexpected sentence is different from an AI system exposing customer information or initiating a privileged action.

A security assessment must therefore follow the attack beyond the initial model response.

Data Exposure

One potential consequence is sensitive information disclosure.

An AI application may receive internal information as part of its context, even when that information is not visible directly through the interface.

This may include private documents, account information, internal instructions, search results, application metadata or responses returned from connected services.

If prompt injection influences what the model chooses to reveal, the AI interface can become an unexpected route around existing information boundaries.

The problem is especially serious when developers retrieve information first and rely on model instructions to decide whether it should be displayed later.

A stronger architecture reverses that logic.

Authorization determines which information may be retrieved.

Only authorized information enters the model context.

The model then works inside the already-approved data boundary.

Tool Manipulation

Modern LLM applications increasingly support tools.

Instead of only generating text, a model may be able to search databases, call APIs, create tickets, interact with files, send communications or control application functions.

Tool access dramatically changes prompt-injection risk.

Once model output can influence a real capability, the application must consider the possibility that an attacker may manipulate the model into requesting that capability.

Anthropic’s research on trustworthy AI agents emphasizes that prompt-injection security requires defenses across multiple layers and advises developers to think carefully about which tools, data, permissions and environments are made available to an agent.

A model should therefore not receive broad tool access simply because certain legitimate workflows occasionally require it.

The safer approach is to restrict tools according to task, user identity and context.

Permission Bypass

One of the most important failure modes occurs when an application treats the AI model as an authorization decision-maker.

Suppose an authenticated user asks an AI assistant to perform an operation.

The model decides which tool to call.

The backend accepts the tool request because it came from the AI.

The architecture has now created a dangerous assumption:

If the model asked for the action, the action must be authorized.

That assumption breaks down under prompt injection.

Backend services should independently determine whether the original user is allowed to perform the requested action.

The model can propose.

The application must authorize.

Prompt Injection in RAG Applications

Retrieval-augmented generation makes AI applications more useful by allowing models to work with external knowledge.

It also creates another route for untrusted instructions to enter the model context.

A RAG system typically receives a question, searches a document store, retrieves relevant information and provides that content to the LLM.

Developers often think of retrieved documents as passive information.

From an adversarial perspective, they are another input channel.

If an attacker can influence documents that may later be retrieved, those documents can potentially contain instructions intended to manipulate the model rather than simply provide information.

This creates several security questions.

Who can add information to the knowledge base?

Can content from different customers enter the same retrieval environment?

Does retrieval enforce user permissions?

Can external websites enter the context?

Can a retrieved document influence tool usage?

The security of RAG therefore depends on much more than vector search quality.

It depends on data provenance, authorization, isolation and how retrieved content is treated after retrieval.

This becomes the central subject of our dedicated article on RAG Security, later in this research series.

Prompt Injection in AI Agents

Prompt injection becomes substantially more consequential when the target system is an AI agent.

An ordinary LLM produces output.

An agent can interpret information, plan additional steps, choose tools and perform actions.

That means a malicious instruction does not necessarily need to produce an immediately dangerous response.

It may instead alter the agent’s objective.

NIST’s work on emerging agentic AI threats describes scenarios where adversaries attempt to steer agents toward objectives embedded in natural-language content, including external documents and retrieval sources.

The security challenge becomes similar to protecting a semi-autonomous software operator.

What information is the agent allowed to trust?

Which tools does it possess?

Which credentials are available?

Which actions require confirmation?

What happens when the agent’s reasoning becomes inconsistent with the user’s original intention?

Agent security therefore requires containment, not only better prompts.

Anthropic has described its own agent-security work in terms of defending multiple layers, including the agent runtime, orchestration system, proxy boundaries and prompt-injection vectors delivered through tools, files or network access.

That architecture-level approach is much closer to conventional cybersecurity than to prompt engineering.

Why System Prompts Alone Cannot Protect an AI Application

System prompts are useful.

They provide behavioral instructions, product context and operational guidelines to an LLM.

But they should not be treated as the final security boundary.

The problem is fundamental: both trusted instructions and potentially hostile content influence the same model.

Adding stronger wording to the system prompt may improve resistance to certain attacks, but it does not transform probabilistic model behavior into deterministic authorization.

OWASP explicitly notes that prompt injection cannot be eliminated through a single perfect prevention mechanism inside the model and instead recommends mitigating the resulting impact through broader controls.

This is similar to traditional security engineering.

Input validation matters.

But input validation does not replace access control.

A firewall matters.

But a firewall does not replace authentication.

A system prompt matters.

But a system prompt does not replace authorization, sandboxing, tool restrictions and data isolation.

Prompt Injection vs Jailbreaking

Prompt injection and jailbreaking are related concepts but are not always identical.

A jailbreak generally focuses on bypassing behavioral restrictions imposed on a model. The objective might be to make the model produce output it would normally refuse to generate.

Prompt injection becomes an application-security issue when manipulated instructions influence the broader behavior of an LLM-integrated system.

The practical distinction is impact.

A jailbreak may affect what the model says.

A prompt injection may affect what the application does.

In real systems the two can overlap, which is why security testing should avoid becoming trapped in terminology.

The relevant question remains:

What security boundary can the attacker cross?

Why Prompt Filtering Is Not Enough

Developers naturally try to identify malicious prompts before the model processes them.

Input screening can be useful.

It can reduce obvious abuse, detect known attack patterns and provide another signal for monitoring.

But relying solely on filters creates a fragile security model.

Natural language is extraordinarily flexible. Equivalent intentions can be represented through different languages, encodings, document structures and contextual strategies.

Even content that appears harmless in isolation may become adversarial when combined with a particular workflow.

Anthropic’s platform guidance recommends layered mitigations rather than treating any individual technique as a complete solution, including input screening, stronger prompt design and careful handling of untrusted tool content.

Security should therefore focus on limiting what happens after a filter fails.

Defensive Controls Beyond Prompt Filtering

A strong prompt-injection defense architecture assumes that some adversarial content will eventually reach the model.

The next question is whether that content can produce meaningful impact.

The most important controls exist outside the model.

Enforce Authorization Outside the LLM

Every sensitive action should remain subject to normal application authorization.

If an AI agent wants to retrieve an account record, the application should determine whether the current user is authorized to access that record.

If the AI requests an administrative action, the backend should evaluate the user’s actual permissions.

Do not substitute:

“The model believes this is appropriate.”

for:

“The authenticated user is authorized to perform this action.”

That single architectural principle can dramatically reduce the consequences of prompt injection.

Apply Least Privilege to AI Tools

Agents should receive the minimum capability required for their current task.

A model used for document summarization should not automatically have account-management privileges.

A customer-support assistant should not receive unrestricted access to every customer database.

A scheduling assistant should not inherit broad mailbox administration simply because it needs calendar access.

The more permissions an agent receives, the larger the potential blast radius of a successful manipulation.

Treat External Content as Untrusted

Webpages, documents, emails, retrieved records and third-party responses should not automatically receive the same trust level as developer instructions.

That sounds obvious when expressed as a cybersecurity principle.

It becomes less obvious when all of those sources are simply converted into text and inserted into the same context window.

Security architects should maintain provenance wherever possible:

Where did this content come from?

Who controls it?

Why is the model seeing it?

Should it be permitted to influence an action?

Sandbox High-Risk Agent Capabilities

Containment limits consequences when preventive controls fail.

Anthropic has publicly described sandboxing as a strategy for limiting what an agent can access even if prompt injection succeeds, illustrating the broader principle that application security should constrain capability rather than assume manipulation can always be prevented.

This is particularly important for agents interacting with filesystems, development environments, browsers and other powerful tools.

Require Confirmation for High-Impact Actions

Some actions should not happen solely because a model selected them.

Deleting information, modifying important records, sending sensitive communications or initiating financially meaningful actions may require deterministic confirmation.

The correct threshold depends on the product.

The principle does not.

As impact increases, model autonomy should become more constrained.

How to Test an Application for Prompt Injection

Effective prompt-injection testing begins with architecture mapping.

Simply sending hundreds of adversarial prompts to an isolated model can demonstrate model behavior, but it tells a security team relatively little about the actual risk of the application.

A proper AI and LLM security assessment first identifies the sources of instructions and data.

Researchers map user inputs, model context, RAG sources, system instructions, memory, connected APIs, available tools and downstream actions.

From that map, security hypotheses can be created.

Could untrusted user content influence the model?

Could external documents introduce instructions?

Could retrieved content cross a tenant boundary?

Could manipulated output reach a privileged tool?

Could an agent perform an action beyond the authority of the user who initiated the workflow?

Testing then evaluates these boundaries under controlled conditions.

The goal is not to produce an impressive jailbreak demonstration.

The goal is to determine whether an attacker can create reproducible security impact.

Test Direct and Indirect Injection Separately

Direct and indirect injection require different testing strategies.

Direct testing examines the channels controlled by users interacting with the system.

Researchers evaluate whether adversarial instructions can alter model behavior and whether that alteration affects sensitive data or application functionality.

Indirect testing examines content the model processes on behalf of the user.

This is especially important for systems that ingest:

web content,

documents,

emails,

support tickets,

knowledge-base entries,

RAG results,

external API responses,

or other machine-generated content.

The security question is whether content controlled by someone other than the legitimate user can influence the AI’s decisions.

For autonomous systems, that may be one of the most important attack surfaces in the entire application.

Validate Security Impact Carefully

An unusual model response is not automatically a severe vulnerability.

Security researchers need to validate what the behavior enables.

The validation path might ask whether the manipulation allows unauthorized information access, causes an unauthorized tool request, affects another user’s data or changes a security-sensitive workflow.

The assessment should generally stop once enough evidence exists to establish impact.

There is little defensive value in causing unnecessary damage after the vulnerable boundary has already been demonstrated.

This is why ShabuShabu’s security testing methodology separates detection from validation and validation from unnecessary exploitation.

Prompt Injection Severity Depends on Agent Capability

The same prompt-injection weakness can have radically different severity across two systems.

System A is a public chatbot connected only to public documentation.

System B is an autonomous agent with access to internal email, customer records and operational tools.

Both models may be manipulated by similar adversarial content.

The consequences are completely different.

This leads to a useful security rule:

Prompt-injection risk grows with model authority.

The more sensitive information, tools and autonomous actions available to the model, the more important containment becomes.

For product teams, reducing unnecessary authority is often more reliable than attempting to eliminate every possible adversarial instruction.

How Organizations Should Think About Prompt Injection Risk

Organizations should avoid asking whether their AI system is “prompt-injection proof.”

That standard is unrealistic.

NIST has repeatedly emphasized that adversarial AI defenses have limitations and that organizations should be cautious about claims of foolproof protection against manipulation.

A more useful set of questions is:

What happens when the model receives adversarial instructions?

What information can the model access?

Which actions can it initiate?

Which decisions are verified outside the model?

Can tools independently enforce user permissions?

Can an attacker influence content used by RAG?

Are dangerous capabilities sandboxed?

Can the organization reconstruct what happened after suspicious agent behavior?

These questions shift AI security from preventing strange model behavior to controlling the consequences of strange model behavior.

That is a much stronger security posture.

Prompt Injection Should Be Tested Before AI Features Reach Production

Prompt injection becomes expensive to fix when broad model permissions are already embedded throughout the architecture.

Testing before deployment gives engineering teams an opportunity to modify trust boundaries before they become operational dependencies.

A pre-launch Security Crash Test is particularly relevant when a product introduces AI agents, external retrieval, confidential datasets or tool-enabled workflows.

Testing is also valuable whenever an existing AI product receives substantially more authority.

Adding a new tool may transform a previously low-impact manipulation into a serious vulnerability.

Connecting a private knowledge base may introduce data-exposure paths.

Increasing agent autonomy may remove a human approval step that previously limited attack impact.

Prompt-injection risk should therefore be reconsidered whenever capability, data access or authority changes.

The Future of Prompt Injection Security

Prompt injection is unlikely to disappear simply because models become more capable.

More capable models may become better at recognizing suspicious instructions, and security tooling will continue to improve.

At the same time, AI systems are gaining access to increasingly valuable tools and environments.

The attack surface is therefore evolving in two directions at once.

Model defenses improve.

Application capability expands.

Anthropic’s 2026 work on trustworthy agents captures this broader principle: prompt injection illustrates why agentic security requires defenses at multiple levels rather than a single model-side safeguard.

For cybersecurity teams, the long-term strategy should therefore resemble mature security engineering.

Assume individual controls can fail.

Limit privileges.

Separate trust domains.

Validate actions.

Contain compromised components.

Preserve meaningful logs.

Test the system adversarially before attackers do.

Prompt injection is a new class of input manipulation, but the strongest defenses are built on security principles that have existed for decades.

Frequently Asked Questions About Prompt Injection

What is a prompt injection attack?

A prompt injection attack occurs when adversarial input influences an LLM to behave differently from the application’s intended instructions. The injected content may come directly from a user or indirectly through information such as webpages, files, emails or RAG documents. OWASP currently classifies Prompt Injection as LLM01:2025.

What is indirect prompt injection?

Indirect prompt injection occurs when malicious instructions are embedded in external content that an AI system later processes. The attacker may control a document, webpage, email or another information source rather than interacting with the target AI application directly. This is particularly relevant for RAG systems and autonomous agents.

Is prompt injection the same as jailbreaking?

They overlap but are not necessarily identical. Jailbreaking generally focuses on bypassing model behavioral restrictions. Prompt injection becomes especially important to application security when manipulated instructions can influence data access, tools, permissions or downstream actions.

Can a system prompt prevent prompt injection?

A strong system prompt may improve resistance but should not be relied on as the sole security control. OWASP notes that prompt injection mitigation requires broader safeguards, while agent-security research increasingly emphasizes layered controls, permissions and containment.

Why is prompt injection dangerous for AI agents?

AI agents can perform actions rather than only produce text. If a manipulated agent has access to internal tools, private data or external systems, prompt injection may influence those capabilities. The potential impact therefore increases as agent authority increases.

How should companies defend against prompt injection?

Defense should combine model-side measures with architecture-level controls. Sensitive data should be authorization-filtered before reaching the model, tools should use least privilege, privileged actions should be independently authorized, untrusted content should be treated as potentially adversarial and high-risk agent capabilities should be contained.

Should prompt injection be included in penetration testing?

Yes, when an application contains LLM or agent functionality. However, effective testing should focus on application-level impact rather than merely demonstrating that a model can be made to produce an unusual response. The assessment should examine data access, RAG, authorization, connected tools and autonomous actions.