When designing LLM services, you’ll soon encounter this question: Should the Application orchestrate everything from user input to RAG retrieval, guardrails, LLM calls, tool execution, and final responses? Or should NeMo Guardrails be at the center, with the Application only handling the beginning and end?
To start with the conclusion, it’s not a matter of designating one as the sole center. The Application or a trusted API layer is the center of security and business control, while NeMo Guardrails Runtime is the center of the LLM processing pipeline. The Application is responsible for authentication, authorization, tenant boundaries, business permissions, final approvals, and audit trails. NeMo orchestrates the Input Rail, Retrieval Rail, Dialog Rail, Execution Rail, LLM calls, and Output Rail according to its configuration.
Therefore, the statement “all network requests must be directly mediated by a single Application process” is not a single official principle from OWASP, Microsoft, or NVIDIA. It is one of several recommended design approaches that can be derived by combining principles such as least privilege, complete mediation, trusted service layers, and data access API encapsulation. What is important is not whether all packets physically pass through the same process, but whether the same authorization policies are consistently enforced across all paths accessing protected data and functions.
Let’s clarify the terminology first
The four terms used in this article are as follows:
- RAG (Retrieval-Augmented Generation): A method that first retrieves external documents related to a user’s query and then inserts those document snippets as reference material into the LLM prompt to generate a response.
- Rail: Programmable policies and flows applied at specific stages of request processing in NeMo Guardrails. They can block inputs, transform retrieval results, control tool execution, and inspect model outputs.
- Tenant Filter: A condition that restricts data retrieval in a multi-tenant environment to only data belonging to the current user’s tenant and within their authorized scope. This involves not just comparing a single tenant_id but also applying security filters such as document classification, user roles, and ownership.
- Presidio: An engine that detects and anonymizes PII (Personally Identifiable Information) such as names, phone numbers, and emails in text. It is not an orchestrator that directs the entire LLM pipeline.
Here, Authentication is the process of verifying “who someone is,” and Authorization is the process of determining “whether that user can use this data or function.” Guardrails enforce allowed LLM behaviors and content policies, but they should not make final business authorization decisions on behalf of the user.
Hub-and-Spoke should be divided into two meanings
The explanation “Application-centric Hub-and-Spoke is secure” is frequently used, but it needs to be clear what is meant by
Leave a Reply