If LLM security is viewed solely as a problem of a single model, it’s easy to miss defense points. In actual services, user input, system prompts, training data, RAG, model output, external tools, and infrastructure are interconnected. Vulnerabilities arise at the boundaries where these components meet, and the damage escalates as they combine.
The Top 10 for LLM Applications 2025, published by the OWASP GenAI Security Project, organizes this entire lifecycle into ten risks. The key is not to memorize the rankings, but to track what data comes from where, and to what authority the model’s output leads.
This article arranges the ten items across input, training, RAG, output, tool, and resource boundaries, then explains what controls should be placed at each boundary. The paths in the tables and figures represent typical points of occurrence, but do not mean that a given risk occurs only at one boundary.
Full List of OWASP LLM Top 10 2025
| ID | Item (English) | Korean Name | Primary Occurrence Boundary |
| — | — | — | — |
| LLM01 | Prompt Injection | 프롬프트 인젝션 | User → System Prompt |
| LLM02 | Sensitive Information Disclosure | 민감 정보 노출 | System/Training → User |
| LLM03 | Supply Chain | 공급망 | Pretraining / 모델 다운로드 |
| LLM04 | Data and Model Poisoning | 데이터·모델 오염 | Training / RAG 데이터 |
| LLM05 | Improper Output Handling | 출력 부적절 처리 | LLM → Downstream |
| LLM06 | Excessive Agency | 과도한 권한 | LLM → Tools/APIs |
| LLM07 | System Prompt Leakage | 시스템 프롬프트 누출 | System Prompt → User |
| LLM08 | Vector and Embedding Weaknesses | 벡터·임베딩 약점 | RAG / Vector DB |
| LLM09 | Misinformation | 잘못된 정보 | Model → User |
| LLM10 | Unbounded Consumption | 자원 무한 소비 | User → Resources |
Figure: OWASP LLM Top 10 Risks and Primary Occurrence Boundaries · Diagram provided by user, used in contrast with OWASP Top 10 for LLM Applications 2025
A notable point in this structure is that while the model is central, most security controls are outside the model. Regardless of what sentences the model generates, data access permissions, command execution, cost limits, and final approvals must be determined by the application and its subsystems.
1. Input and Instruction Boundary: LLM01, LLM07, LLM02
LLM01 Prompt Injection — Input changes instructions
Prompt Injection is a risk where user input or external documents alter the model’s intended behavior. This includes not only direct attacks where users explicitly write “ignore previous instructions,” but also indirect attacks where the model reads hidden instructions embedded in webpages, emails, PDFs, or images. RAG and fine-tuning can improve the relevance of responses but do not completely eliminate this risk.
Therefore, a single input filter should not be considered the entirety of defense. Untrusted content must be separated from system instructions, and separate authorization boundaries should be established to prevent attacks on the model from leading to sensitive data queries or external task execution.
LLM07 System Prompt Leakage — System prompts are not secret storage
The core of System Prompt Leakage is not about permanently hiding the prompt text itself. OWASP explains that system prompts should not be treated as secrets or security controls. If API keys, connection strings, internal accounts, or authorization rules are placed in the prompt, the real problem is not “the prompt was leaked,” but rather that secrets and permissions were placed where they shouldn’t be.
Secrets should be stored in KMS or secret vaults, and decisions like authentication, authorization, and transaction limits should be enforced by code and policy engines. It is safer to leave only information guiding the model’s role and response format in system prompts.
LLM02 Sensitive Information Disclosure — Reduce data paths before it appears in responses
Sensitive Information Disclosure is the risk of personal information, financial/medical data, trade secrets, credentials, training data, or application context being revealed through responses. A system prompt like “Do not disclose sensitive information” is insufficient, as it can be bypassed by prompt injection or model errors.
During the input stage, sensitive data should be masked; during the retrieval stage, only documents matching user permissions should be queried; and during the output stage, DLP and policy checks should be reapplied. The strongest control is to avoid providing unnecessary sensitive information to the model in the first place.
2. Supply Chain and Training Boundary: LLM03, LLM04
LLM03 Supply Chain — AI supply chain broader than code dependencies
Supply Chain risk includes not only general libraries but also pre-trained models, datasets, LoRA adapters, model repositories, deployment platforms, and licenses. Untrusted model files may contain biases or backdoors, and the serialization format or loading code itself could execute malicious actions.
The origin, version, hash, signature, and license of models and data must be recorded and verified. In addition to SBOMs, a BOM that tracks AI/ML components and data sources should be maintained, and external models should undergo evaluation and red team validation tailored to actual usage scenarios.
LLM04 Data and Model Poisoning — Integrity issues in training and knowledge bases
Data and Model Poisoning is the risk where pre-training, fine-tuning, embedding data, or the model itself is manipulated to create biases, backdoors, performance degradation, or harmful outputs. Trigger-type backdoors, which appear normal but activate under specific phrases or conditions, are difficult to detect with standard quality tests alone.
Data sources and transformation history should be preserved, and change approval, version control, anomaly detection, and behavioral evaluation should be incorporated into the pipeline. RAG documents also change continuously during operation, so their source and integrity must be managed at the same level as training data.
3. RAG and Vector Store Boundary: LLM08
Vector and Embedding Weaknesses cover authorization errors, information leakage between tenants, retrieval of poisoned documents, embedding inversion, and knowledge conflicts that occur during the creation, storage, and retrieval of embeddings. Attaching RAG does not automatically make responses safe. If poisoned documents are retrieved more accurately, or if another user’s documents are placed in the context, the attack path actually shortens.
Authorization checks should be applied at the search query stage, not after retrieval and handed over to the model. Partitions should be created by user, organization, and document classification, and only results with applied authorization filters should be passed to the model. The document collection pipeline should include source authentication, hidden text inspection, malicious content quarantine, and change auditing.
4. Output and Subsystem Boundary: LLM05, LLM09
LLM05 Improper Output Handling — Model output is also untrusted input
Improper Output Handling occurs when a string generated by the model is passed to a browser, shell, SQL, file path, email template, or another API without validation. Since model output is influenced by user input, it is essentially another form of untrusted input from the perspective of downstream systems.
For HTML, context-specific output encoding and CSP should be applied; for SQL, parameterized queries should be used; and shell commands should be restricted to allowed structures. Tool calls should not execute free-form text but should only accept structured arguments that have passed strict schema validation and policy checks.
LLM09 Misinformation — Plausibility and factual accuracy are different
Misinformation is the risk that a model generates incorrect or unfounded content in a plausible manner. Beyond hallucinations, biased training data, outdated or incomplete information, and user overconfidence also amplify the impact. Even without an attacker, automatically processing incorrect answers in legal, medical, financial, or security judgments can lead to harm.
RAG based on reliable sources, source attribution, automatic validation of key fields, and expert review of high-risk results are necessary. Do not stop at merely indicating “AI-generated”; clearly communicate in the user interface what uses are permitted and who is responsible for verification.
5. Tool and Resource Boundary: LLM06, LLM10
LLM06 Excessive Agency — Reduce functionality, permissions, and autonomy separately
Excessive Agency is the risk where a model or agent possesses more functionality, permissions, or autonomy than necessary, leading to unexpected operations. Examples include granting send/delete permissions to an email summarization feature, using database modification permissions for a query task, or performing high-risk operations without user approval.
The list and functionality of tools should be minimized, and restricted by user-specific OAuth scopes and the actual permissions of downstream systems. High-impact operations like transfers, deletions, external dispatches, and deployments should require explicit human approval immediately before execution. Authorization decisions must be re-checked by the downstream system for every request, not based on the model’s statements or system prompts.
LLM10 Unbounded Consumption — Token cost is also a security boundary
Unbounded Consumption is the risk of denial of service, cost exhaustion, performance degradation, or model extraction due to excessive input, repeated requests, long agent loops, or costly tool calls. In a pay-as-you-go cloud inference structure, “Denial of Wallet,” where billing costs surge even if the server doesn’t crash, becomes a realistic impact.
Request rates and cost limits should be set per user, organization, and API key. Input/output tokens, execution time, number of tool calls, maximum agent steps, and queue length should be limited. Monitor for anomalies in cost and latency, and prepare for graceful degradation, where functionality is reduced instead of a full outage if limits are exceeded.
Risks don’t come one by one
In real-world incidents, multiple items appear in sequence. For example, if an attacker inserts hidden instructions into a RAG document, LLM04 Data Poisoning and LLM08 Vector and Embedding Weaknesses become the starting points. If the retrieved instructions alter model behavior, it leads to LLM01 Prompt Injection. If unvalidated output is passed to a tool, LLM05 comes into play, and if the tool executes with excessive agency, LLM06 expands the damage. If the result is sensitive data leakage, LLM02 is also connected.
Therefore, simply increasing “prompt injection detection rates” alone cannot address the entire risk. Independent controls must be layered so that even if an attack bypasses one layer, it is stopped at the next.
How to deploy controls across six boundaries
Figure: Reconfiguring OWASP LLM Top 10 into six defense boundaries · Reconfigured based on prevention and mitigation items from OWASP 2025 LLM Top 10
| Boundary | Primary Controls | Mainly Reduces Risks |
| — | — | — |
| Supply Chain | Supplier verification, model/data hash and signature, BOM, license records | LLM03, LLM04 |
| Training/Collection | Data source/version management, anomaly detection, change approval, behavioral evaluation | LLM04, LLM02 |
| Input/Instruction | Untrusted content separation, input validation, sensitive data masking, attack simulation | LLM01, LLM07, LLM02 |
| RAG/Memory | Pre-search authorization filter, tenant separation, document quarantine, search logs | LLM08, LLM04, LLM02 |
| Output/Tools | Schema validation, context-specific encoding, least privilege, user approval | LLM05, LLM06, LLM09 |
| Resources/Operations | Rate limit, token/cost/step limits, timeout, anomaly detection | LLM10, LLM06 |
In practice, it’s good to start by drawing a data flow diagram and noting the trust level, data classification, calling entity, authentication method, maximum cost, and default behavior on failure for each arrow. Connections without this information often lack security requirements.
Four frequently confused pairs of items
LLM01 and LLM07
LLM01 is an attack method where input changes the model’s behavior, while LLM07 is the risk of system prompt content being exposed to the user. Although system prompt extraction can occur via prompt injection, the focus of the two items is different.
LLM05 and LLM06
LLM05 is a problem of insufficient validation, encoding, or handling of model output. LLM06 is a problem of excessive functionality, permissions, and autonomy available to the model. Even with output validation, if an administrator’s permission is given to a deletion tool, LLM06 remains.
LLM03 and LLM04
LLM03 is a supply chain perspective, focusing on the origin and dependencies of external providers, models, libraries, and datasets. LLM04 focuses on the phenomenon where data or models are actually manipulated, compromising integrity.
LLM08 and LLM09
LLM08 concerns access control, retrieval, embeddings, and data integrity issues in RAG and vector stores. LLM09 concerns the factual accuracy of the final answer and user overconfidence. Even if search permissions are accurate, if the model misinterprets the evidence, LLM09 can occur.
Minimum Checklist Before Deployment
- Are API keys, credentials, or actual authorization rules present in the system prompt?
- Are user input and external documents clearly separated from system instructions?
- Are user, tenant, and document-level permissions applied before RAG retrieval?
- Are the source, hash, version, and license of models, LoRA, datasets, and libraries tracked?
- Is model output re-validated to match HTML, SQL, shell, file path, and tool schemas?
- Do tools have only necessary functionality and permissions, and do high-risk operations require human approval?
- Are important answers subject to source links, automatic validation, or expert review?
- Are there upper limits on request rate, tokens, cost, execution time, agent steps, and number of tool calls?
- Can attack inputs, poisoned documents, incorrect searches, tool failures, and cost spikes be observed via logs and alerts?
Conclusion
The principle running through the OWASP LLM Top 10 2025 is simple: do not use the model as a trust boundary; treat both the model’s input and output as untrusted data; and control actual permissions and costs outside the model.
Instead of memorizing the ten items separately, arranging them on a single data flow diagram—Supply Chain → Training → Input/RAG → Model → Output → Tools/Resources—makes it easier to spot missing controls. Only when prompt defense, data integrity, permission-based retrieval, output validation, least privilege, and resource limits work together can the overall risk of LLM applications be reduced.
Leave a Reply