Assembling the Defense Line: Serial Guardrails and Control Plane

A serial guardrail is a structure that allows a single request to pass through multiple inspection gates in a predefined order. The Control Plane is not the path that directly passes through those gates, but rather a control layer that deploys which policies and models to execute in what order and manages their versions.

In previous exercises, we examined NeMo Guardrails, Nova Lite-based Safety classification, and Microsoft Presidio. Now, the important question is not “how many tools have been enabled,” but “who determines the order, how failures are aggregated, and how policy changes are tracked.” In this article, we will create a serial Data Plane with synthetic test cases and add a small Control Plane contract.

User requests pass through multiple security gates in series, and the upper Control Plane manages policies and states

First, distinguish between Data Plane and Control Plane

The Data Plane is the execution path through which actual user requests, retrieved documents, and model responses flow. The Control Plane manages policy bundles, prompt versions, model settings, thresholds, and deployment states. In the event of a Control Plane failure, unverified new policies should not be immediately downloaded, nor should all requests be allowed.


Category	Responsibilities	What not to handle

—	—	—
Data Plane	Request inspection, RAG refinement, LLM/Tool execution, Output inspection	Arbitrary policy changes and unapproved deployments
Control Plane	Policy Version, Approval, Signature/Hash, Deployment, Rollback	Raw user requests and final business authority
Application	Authentication, Tenant, Business authority, Final approval, Audit	Delegating access control decisions to LLM
NeMo Runtime	Rail/Action/Retrieval/LLM flow orchestration	Sole determination of final user authorization

Here, Control Plane does not mean a separate mandatory component of NVIDIA products. NeMo official documentation describes config.yml, Prompt, Colang Flow, and Runtime Orchestration. A central policy repository and approval/deployment procedures are this article’s recommended design for operating them.

Recommended Serial Flow

Browser
  ↓
Application: 인증·Tenant·요청 한도
  ↓
Nova Lite 분류: 명시적 Safety Category
  ↓
NeMo Input Rail: Self-check·Jailbreak·주제 정책
  ↓
Retrieval API: Tenant Filter와 Security Trimming
  ↓
NeMo Retrieval Rail: Presidio로 원문 Chunk 정제
  ↓
Dialog·Execution Rail: 대화 흐름과 Tool 호출 검증
  ↓
Main LLM
  ↓
Output Rail: Safety·PII·형식 검사
  ↓
Application: 최종 업무 권한·감사·응답

NVIDIA’s official documentation explains that the Input Rail inspects input before the Main LLM, the Retrieval Rail processes retrieved documents and chunks before they are inserted into the Prompt, the Execution Rail controls Actions and Tools, and the Output Rail inspects responses before they are returned. The exact execution order may vary depending on the chosen Engine and configuration.

The advantage of a serial structure is that blocking at an earlier stage can skip expensive subsequent calls, and it’s easy to explain which gate made the decision. The disadvantages are that the latency of each stage adds up, multiple policies might redundantly block the same request, and a failure in an earlier stage can affect overall availability.

Practice 1: Creating a Common Judgment Contract

If each tool returns results in a different format, the integration code can easily convert exceptions into allowances. First, ensure all Adapters return the same format.

from dataclasses import dataclass, field
from typing import Literal

Decision = Literal["ALLOW", "BLOCK", "MODIFY", "ERROR"]

@dataclass
class GuardrailResult:
    decision: Decision
    stage: str
    rule_id: str
    policy_version: str
    sanitized_text: str | None = None
    labels: list[str] = field(default_factory=list)

Do not copy user original text or detected personal information into the reason. Operational records should only include Stage, Rule ID, Decision, Policy Version, Latency, and Correlation ID.

Practice 2: Assembling a Serial Pipeline

The code below is a skeleton for connecting asynchronous Adapters with identical interfaces instead of actual Providers. Insert Nova, NeMo, and Presidio calls inside each function.

import asyncio

class Blocked(Exception):
    def __init__(self, result: GuardrailResult):
        self.result = result

async def enforce(result: GuardrailResult, text: str) -> str:
    if result.decision in {"BLOCK", "ERROR"}:
        raise Blocked(result)
    if result.decision == "MODIFY":
        if result.sanitized_text is None:
            raise Blocked(GuardrailResult(
                "ERROR", result.stage, "missing_sanitized_text",
                result.policy_version,
            ))
        return result.sanitized_text
    return text

async def run_pipeline(user_text: str, tenant_id: str) -> str:
    text = await enforce(await nova_classify(user_text), user_text)
    text = await enforce(await nemo_input_check(text), text)

    chunks = await authorized_retrieve(
        query=text,
        tenant_id=tenant_id,
    )
    safe_chunks = await presidio_sanitize_chunks(chunks)

    tool_plan = await nemo_dialog_and_execution(text, safe_chunks)
    answer = await call_main_llm(text, safe_chunks, tool_plan)
    answer = await enforce(await nemo_output_check(answer), answer)
    return answer

This example blocks ERROR by default. While availability-critical services like read-only FAQs can switch to a restricted mode by removing RAG and Tools, for agents handling sensitive information or write permissions, fail-closed is a safe default.

Practice 3: Making NeMo Configuration Order Visible

In NeMo config.yml, active Rail Flows are declared. Official configuration documentation states that Input and Output Flows can be listed in sequence, and some environments also support parallel execution.

rails:
  input:
    parallel: false
    flows:
      - self check input
      - check jailbreak
      - mask sensitive data on input

  retrieval:
    flows:
      - check retrieval sensitive data

  output:
    parallel: false
    flows:
      - self check output
      - check output sensitive data

tracing:
  enabled: true
  enable_content_capture: false

For Rails that modify input, the result of the next inspection changes depending on the order. NVIDIA’s official documentation also warns that parallel execution of Input Rail Mutation can lead to different results due to race conditions, recommending serial execution in such cases. While independent read-only classifiers can be parallelized, regression tests are needed to ensure that their meaning is not altered for performance reasons.

Furthermore, NeMo Tracing’s Message Content Capture is disabled by default. If raw content capture is enabled in production, Prompts, responses, and Tool results may be duplicated in the trace, so it should not be activated without separate approval, retention period, and access control.

Practice 4: The Smallest Control Plane Contract

The Control Plane is not a download server that allows execution code to receive arbitrary latest files. It provides the Version and Hash of an approved Policy Bundle, and the Data Plane only activates bundles verified at startup.

bundle_id: llm-guardrails-prod
version: 2026.09.03-1
status: approved
minimum_runtime: "0.18"
default_failure_mode: fail_closed

stages:
  - id: nova-input-safety
    policy: policies/nova-input-v3.txt
    timeout_ms: 1200
  - id: nemo-input
    policy: configs/nemo/config.yml
    timeout_ms: 1800
  - id: presidio-retrieval
    policy: policies/pii-entities-v5.yml
    timeout_ms: 500

artifacts:
  manifest_sha256: "배포 과정에서 계산한 값"

Do not include actual Secrets, API Keys, or personal information samples in the Manifest. Bundles should be created as Immutable Artifacts via Git Pull Requests or an approval workflow, and the Data Plane should verify the allowed signers, Hash, and Runtime Version.

Practice 5: Creating Deployment and Rollback States

Policy deployment should be treated like code deployment.

draft → reviewed → approved → canary → active
                     ↘ rejected
active → rollback_previous

In Canary, compare normal allowance rates, attack blocking rates, timeouts, P95 latency, and per-stage costs using synthetic test traffic. If the criteria are not met, roll back to the previous Bundle. Instead of assuming “the latest policy is safe,” leave comparable evidence with the previous version.


Change	Required Verification

—	—
Safety Prompt	Normal/attack classification regression, output format
Presidio Recognizer	False Positive/False Negative, language variations
Rail Order	Results before and after mutation, occurrence of subsequent calls
Model Replacement	Policy compliance, Latency, Cost, Timeout
Tool Schema	Verification of allowed functions, arguments, and results
Failure Mode	Blocking or restricted mode upon fault injection

Minimum Signals to Observe

For each request, record the following values as structured logs and trace attributes:

request_id, non-identifiable internal identifier of the validated tenant_id
policy_bundle, policy_version, runtime_version
stage-specific decision, rule_id, latency_ms
type and success status of executed LLM, Retriever, Tool
final blocked_by and whether in restricted mode

Prompt and response raw text, PII, and Credentials are excluded from default telemetry. User input or unique identifiers should not be included in metric labels. Traceability comes not from storing a lot of raw text, but from linking policy versions and execution paths with the same Request ID.

Choosing Serial, Parallel, or Hybrid


Method	Advantages	Risks	Suitable Cases

—	—	—	—
Fully Serial	Clear order and blocking reason	Cumulative Latency, upstream bottleneck	Mutation, sensitive RAG, Tool Agent
Fully Parallel	Potential for reduced independent inspection Latency	Race conditions, redundant costs, integration complexity	Read-only independent classifiers
Hybrid	Preserves semantic order while optimizing some parts	Design/observability complexity	Services with growing operational scale

The recommended starting point is serial. After measurement results accumulate, only independent inspections that do not modify each other’s inputs should be parallelized. If one of the parallel calls allows a request first, the Main Response should not be returned immediately; the final combined policy must wait for all required judgments.

Practice Completion Checklist

Separated the responsibilities of Data Plane and Control Plane.
Authentication, Tenant, and business authority are left to the Application and Downstream.
All Guardrail Adapters return a common judgment contract.
Documented the execution order and short-circuit rules of the Rails.
Defined default behaviors for errors, timeouts, and unknowns.
Recorded Version, Hash, approval status, and Runtime conditions in the Policy Bundle.
Prepared Canary and previous Version Rollback paths.
Excluded Prompt, PII, and Credentials from default Telemetry.
Executed synthetic regression tests for every policy, model, and order change.
Verified with traces that no RAG, Tool, or Main LLM calls occur after blocking.

Conclusion

A serial guardrail is not about simply stringing together multiple security products. It’s about clearly defining the inputs and outputs, modification rights, failure policies, and ultimate decision-makers at each stage. The Application handles authentication, authorization, and business control, NeMo orchestrates the LLM processing flow, and inspections like Nova and Presidio provide specialized judgments within defined boundaries.

The Control Plane manages policy versions, approvals, deployments, and rollbacks over this execution path. First, ensure accuracy with a simple serial path, and only parallelize independent inspections when there is measured evidence to do so. In the next part, we will verify that policy changes do not break existing defenses by repeatedly executing known attacks and normal requests with Promptfoo.

Official Resources

※ The Control Plane Bundle and approval/deployment states are not official product requirements but are recommended operational design proposals in this article. NeMo API and Configuration may vary by Version, so please verify with fixed Release documents and synthetic test cases.