LLM01 Prompt injection |
02 · 03 |
Text addressing the model rather than the reader. Changes of role, rules being cancelled, or imitation system markers inside a document. |
Narrow credentials below the model, clear marking of retrieved text, and limits on what can be sent outward. Classifiers help at the margins, never on their own. |
LLM02 Sensitive information disclosure |
02 · 03 · 05 |
Requests that quietly broaden the scope: everything you know about, the complete record, the raw context. |
Do not place in the prompt anything the user is not entitled to read. Filter by identity when the search runs, rather than afterwards. |
LLM03 Excessive agency was LLM06:2025 |
04 · 06 |
An agent able to do considerably more than its task requires. Broad credentials, permanent access, and no confirmation on actions that cannot be reversed. |
Issue credentials for one task only. Offer one list of tools per step. Require a person to approve anything irreversible. |
LLM04 Supply chain was LLM03:2025 |
01 |
Unrecorded model versions, community-supplied adapters, a plugin or connected server nobody reviewed, or a pickle file. |
Record versions. Prefer the safetensors format. Review tool manifests as you would review code. Treat a connected server as a dependency with write access. |
LLM05 Data and model poisoning was LLM04:2025 |
01 · 03 |
Content written specifically to be retrieved. Documents that read oddly to a person but match a target question unusually well. |
Control who may write into the store. Clean documents as they arrive rather than as they leave. Record the origin of every chunk. |
LLM06 Unbounded consumption was LLM10:2025 |
01 · 04 |
A single request that expands into hundreds of calls. Loops without a ceiling, very long documents, or agents calling further agents. |
Set a budget for each request covering tokens, steps, elapsed time and money, and enforce a hard stop on the loop. |
LLM07 Misinformation was LLM09:2025 |
07 |
Answers that are fluent, confident and unsourced. Invented citations, and package names that do not exist. |
Display the source beside the claim. Verify citations mechanically. Design the interface so that uncertainty remains visible. |
LLM08 Hidden context exposure was LLM07:2025 |
02 |
Probing for anything the application holds but never displays: the instructions, the retrieved documents, the tool list, or earlier turns. |
Assume it will leak. Place no secret, credential or permission rule anywhere in the prompt, and enforce those in code where they cannot be argued with. |
LLM09 Vector and embedding weaknesses was LLM08:2025 |
03 |
A search returning documents belonging to another customer, or a single chunk that matches every question. |
Separate customers at the store rather than in the prompt. Filter before the search runs, then confirm ownership again afterwards. |
LLM10 Improper output handling was LLM05:2025 |
05 |
Model output passed directly into a browser, a command shell, a database query or another agent without being escaped. |
Escape it for wherever it is going, and permit only known destinations. This is ordinary web security and the least expensive improvement on the list. |