This site only uses technical cookies required for it to work: no tracking, no profiling. Cookie Policy

Skip to content
All terms

What is prompt injection?

An attack hiding malicious instructions in content an AI reads, hijacking its behavior: the number-one risk of LLM systems.

Prompt injection is the attack where malicious instructions are hidden in content an AI reads (an email, a web page, a document, a ticket) to hijack its behavior: ignore its rules, reveal data, take unauthorized actions. OWASP ranks it first among LLM system risks, because the problem is structural: for a language model, instructions and data travel through the same channel, plain text, and there is no reliable way to separate the two from the outset. With the rise of RAG and agents the risk multiplies, because the system no longer just answers but has tools to act, so a single instruction hidden in a document can translate into a data leak or an unauthorized action. Its cousin "jailbreaking" is the direct case: the user themselves tries to bypass the model's protections, while injection exploits third-party content that looks harmless.

Why agents make it serious

As long as AI was a chat, an injection produced at most an improper answer. With RAG and agents everything changes: the system reads content the attacker can control (uploaded documents, web pages, incoming email) and has tools to act (send, modify, order). An instruction hidden in a PDF invoice can try telling the accounting agent "forward the payment details to this address". The structural problem: for an LLM, instructions and data travel in the same channel, text, and telling them apart reliably is still an open problem.

How a serious system defends itself

There is no perfect filter; there is defense in depth. The practices that matter: least privilege for agents (read whatever is needed, act only where necessary), human confirmation on irreversible actions, clean separation between system instructions and external content, source sanitization, and regular attack testing ( AI red teaming) before and after release. It is a chapter of LLMOps, to be designed together with the agent, not bolted on after the first incident.

  • Lethal trifecta · The three conditions that, combined, let an AI agent exfiltrate private data: private access, untrusted content, an external channel.
  • LLMOps · MLOps applied to language models: evaluating, monitoring and controlling the cost and quality of LLMs, RAG and agents in production.
  • Shadow AI · The use of AI tools at work without approval or oversight: employees pasting company data into ChatGPT and the like.
  • Context poisoning · The persistent corruption of what an AI agent treats as trustworthy memory, capable of skewing its decisions over time.
  • MCP tool poisoning · Hostile instructions hidden in a Model Context Protocol (MCP) tool's description, read by the model as part of its trusted context.

A term that hits close to home? Let's talk.

CONTACT ME