Prompt injection

Description

The direct variant is a user in conversation trying to make the model ignore its rules. The indirect variant is more dangerous: the malicious instruction sits hidden in a document, a web page or an email the agent reads. The victim often never sees the instruction, but the model will process it.

The damage depends on the permissions the agent holds. If it only answers, it can give away data it has access to. If it may send mail, call an API, change a ticket, read files or delete data, the attack is no longer a game with text but an operational incident.

Examples

• An agent summarizing email reads hidden instructions to forward the latest messages to an external address.
• A chatbot connected to a database receives a request to ignore its previous rules and print another user's data.
• A PDF tender contains hidden text that tries to change the conclusion of the assistant comparing the bids.

Notes

• With LLM agents the problem is not only what the model says but what the model is allowed to do.
• Before any debate about the perfect prompt, the permissions need narrowing, the data separating, and actions with consequences need a confirmation step.