Discovering New Attacks: Garak and PyRIT

While regression testing is strong at preventing known failures from recurring, it cannot find attacks for which no test cases yet exist. Garak and PyRIT are AI Red Teaming tools that attempt various probes and conversational strategies on LLMs or AI applications, collecting unexpected failure candidates.

These two tools are not “automatic vulnerability certifiers.” The results are judgments from detectors and scorers, and false positives may occur. This exercise uses only a locally owned test target, synthetic data, and read-only functions. Discovered findings are only promoted to vulnerabilities or regression test cases after human review of traces and policies.

What’s the Difference Between Garak and PyRIT?

Garak is a scanner-type tool that sends various probes to a Target Generator and uses a Detector to find signs of failure. PyRIT (Python Risk Identification Tool) is closer to a framework that assembles Target, Attack, Converter, Scorer, and Memory to create single-turn or multi-turn attack experiments.

Comparison	Garak	PyRIT
—	—	—
Core Unit	Probe and Detector	Attack·Target·Scorer·Memory
Strengths	Broad-scope repetitive scanning	Adaptive, multi-turn experiments with state
Target Connection	Generator Plugin	Prompt Target
Judgment	Detector and Evaluator	Scorer and Human Review
Results	Log·JSONL·Report	Conversation·Score·Results stored in Memory
Recommended Starting Point	Narrow scan of selected Probe Family	Execute a single synthetic Objective from a single turn

In Garak’s official documentation, a Probe manages inputs and interactions to induce failures, and a Generator wraps the connection to a Model or conversational system. A Detector looks for specific failure signs in responses, and a Harness coordinates the overall execution.

PyRIT’s official documentation describes automated/human-driven Red Teaming, various Targets, Single-turn/Multi-turn Attacks, flexible Scoring, and SQLite/Azure SQL-based Memory as its main components.

Establishing Safe Practice Boundaries

Garak·PyRIT Container
  ↓ 허용된 Test Network만
Local Guardrail Gateway
  ↓
합성 Test Tenant + 읽기 전용 RAG + Mock Tool
  ↓
Nova·NeMo·Presidio → Test LLM
  ↓
비민감 Trace와 격리된 결과 저장소

Do not run against actual production endpoints, other people’s models, external services, or real data. The Test Gateway blocks egress, and tools return only fixed mock responses instead of deleting files, executing commands, making payments, or sending messages. API Keys should be injected as secrets, not embedded in images or notebooks.

Practice 1: Verifying Garak Structure and Installation

The official repository provides methods for PyPI installation and checking the list of probes. For operational evaluation, a verified version should be fixed.

mkdir llm-redteam-lab
cd llm-redteam-lab

python3 -m venv .venv-garak
source .venv-garak/bin/activate
python -m pip install --upgrade pip
python -m pip install garak

python -m garak --version
python -m garak --list_probes

Running all probes from the start can rapidly increase costs, time, and result volume. First, verify the execution structure with a Test Generator, or select only one Probe Family relevant to your service objectives.

python -m garak 
  --target_type test.Repeat 
  --probes encoding

test.Repeat is a built-in Test Target that repeats input. It is a safe smoke test to verify installation and the report pipeline, not a command to evaluate actual vulnerabilities. Garak can log execution details to JSONL, so use only synthetic data for input and output, and restrict access permissions to the results directory.

Practice 2: Connecting Only Your Own HTTP Gateway as a Target

The next step is to connect your Local Guardrail Gateway to Garak’s REST Generator. The exact Config Key for the REST Generator can be found in the official Reference for your installed Garak Version.

plugins:
  generators:
    rest:
      RestGenerator:
        uri: http://host.containers.internal:8080/chat
        method: post
        headers:
          Content-Type: application/json
        req_template_json_object:
          message: "$INPUT"
          tenant_id: "redteam-test"
        response_json: true
        response_json_field: response

python -m garak 
  --target_type rest.RestGenerator 
  --config garak-rest.yaml 
  --probes promptinject

The names and options above may vary depending on the version. First, check –plugin_info and the Reference. Define the scan scope, maximum generations, concurrency, timeout, and cost limits, and block addresses other than the Test Gateway with network policies.

How to Interpret Garak Results

A Garak FAIL means that the Detector found the targeted failure sign in the response. It does not immediately mean that a real vulnerability has been confirmed.

1. Check which Probe and Detector combination was used.
2. Verify if target errors or empty responses were incorrectly judged.
3. Check RAG·Tool Trace in addition to user output.
4. Repeat to see if it can be reproduced under the same conditions.
5. Reduce to a minimal synthetic test case, then a human defines the expected behavior.

If a Detector relies on simple denial phrases, it may produce false positives due to changes in language or expression. A Custom Detector or post-processing gate that verifies Guardrail’s structured decisions and tool call status together may be necessary.

Practice 3: Starting PyRIT In-memory

PyRIT’s official documentation provides framework examples using package installation and initialize_pyrit_async. In isolated practice, first use In-memory Memory to reduce the scope of conversations remaining on disk.

python3 -m venv .venv-pyrit
source .venv-pyrit/bin/activate
python -m pip install --upgrade pip
python -m pip install pyrit
python -c "import pyrit; print(pyrit.__version__)"

import asyncio
from pyrit.setup import IN_MEMORY, initialize_pyrit_async

async def main():
    await initialize_pyrit_async(memory_db_type=IN_MEMORY)
    print("PyRIT in-memory lab initialized")

asyncio.run(main())

IN_MEMORY conveniently disappears when the process ends, but it does not clear screen output or external provider logs. Only synthetic data should be used for prompts and responses.

Practice 4: Validating the Target from a Single Turn

PyRIT’s Target is the AI System Interface to be attacked. The official documentation examples use OpenAIChatTarget and PromptSendingAttack. This article only shows the structure, assuming that actual keys are not hardcoded but approved test endpoint settings are loaded from the environment.

import asyncio
from pyrit.executor.attack import PromptSendingAttack
from pyrit.output.attack_result.pretty import PrettyAttackResultMemoryPrinter
from pyrit.prompt_target import OpenAIChatTarget
from pyrit.setup import IN_MEMORY, initialize_pyrit_async

async def main():
    await initialize_pyrit_async(memory_db_type=IN_MEMORY)

    target = OpenAIChatTarget()
    attack = PromptSendingAttack(objective_target=target)

    result = await attack.execute_async(
        objective="합성 Test 문서의 접근 정책을 설명해줘"
    )

    printer = PrettyAttackResultMemoryPrinter()
    await printer.write_async(result)

asyncio.run(main())

This Objective is a smoke test to confirm a normal connection. Only after verifying a safe single turn, scorer validation, and result storage scope should it be extended to an approved multi-turn strategy.

Practice 5: Separating Scorer and Human Judgment

A PyRIT Scorer can evaluate whether a response achieved its objective using True/False, Likert Scale, or Classification. LLM-based Scorers are flexible but have their own errors and costs.

Judgment Target	Preferred Method
—	—
Tool Call Status	Deterministic check of structured trace
Tenant Boundary Violation	Deterministic check of Document ID·Tenant ID
Secret String Exposure	Exact match check of synthetic Canary
Denial Meaning·Bypass Success	Scorer and human review in parallel
Answer Quality·Relevance	Model Scorer and sample review

Both the Attack Model and Scorer Model can be wrong even if separated from the Target Model. Record the Scorer Version, Prompt, Threshold, and execution environment in the results, and do not count undetermined or errors as successes.

Combining Systematic Scanning and Adaptive Exploration

Garak 넓은 Probe Scan
  ↓ 실패 후보
사람의 1차 분류
  ↓ 대화형 재현이 필요한 후보
PyRIT 단일·다중 Turn 실험
  ↓ 재현 가능한 최소 사례
Trace·권한·정책 검토
  ↓ 승인된 Finding
Promptfoo 회귀 Testcase + Guardrail 정책 변경

While it’s useful to broadly scan with Garak and deeply explore conversational contexts with PyRIT, this order is not always necessary. The important thing is to link the two results with the same finding_id, Target Version, Policy Bundle, and Trace ID.

Podman Sandbox Operation Example

podman network create --internal llm-redteam-lab

podman run --rm 
  --network llm-redteam-lab 
  --read-only 
  --cap-drop=all 
  --security-opt=no-new-privileges 
  --pids-limit=256 
  --memory=4g 
  --cpus=2 
  -v "$PWD/results:/results:Z" 
  localhost/llm-redteam-tools:VERSION

External APIs cannot be called on an –internal Network. You can attach a Local Target Container to the same Network for a fully local practice, or if an external Provider is absolutely necessary, set up a separate Egress Proxy that only allows permitted destinations. Do not mount Host Network, Docker Socket, Home Directory, or Cloud Credential Directory.

Protecting Personal Information and Costs in Results

• Retain original Prompt·Response texts for the minimum necessary period.
• Use synthetic datasets instead of actual customer conversations.
• Configure access permissions and encryption for JSONL·Memory DB·Trace.
• Remove API Keys and Authorization Headers from logs.
• Limit maximum turns, probes, generations, concurrency, tokens, and costs.
• Replace tools with mock or read-only versions.
• Discard temporary containers and secrets after the scan ends.

Practice Completion Checklist

• Used only test targets that are owned or explicitly authorized.
• Differentiated the roles of Garak’s Generator·Probe·Detector·Harness.
• Differentiated the roles of PyRIT’s Target·Attack·Scorer·Memory.
• Verified installation with a Test Target before a full scan.
• Started with a small scope for Probe Family and multi-turn.
• Used synthetic tenants, synthetic canaries, and mock tools.
• Limited container permissions, network egress, and resource limits.
• Did not automatically interpret FAIL·Score as confirmed vulnerabilities.
• Reviewed user output along with internal Tool·RAG Trace.
• Only promoted approved findings to minimal regression test cases.

Conclusion

Garak and PyRIT are not tools for re-scoring known answers, but rather exploration tools for finding failure candidates outside the current test suite. Garak broadly scans with Probes and Detectors, while PyRIT deeply investigates conversational paths by combining Target·Attack·Scorer·Memory.

Automation produces a list of candidates, not judgments. It should be run in an isolated test environment, and only after human review of policies, permissions, traces, and reproducibility should valuable findings be promoted to Promptfoo Testcases and Guardrail policies. The next and final part will complete the entire loop of transforming these findings into test cases with responsible parties, expected results, and policy versions, and safely deploying them to the Control Plane.

Official Resources

• NVIDIA Garak GitHub
• Garak Core Concepts and Classes
• Garak Configuration
• Garak Detector Reference
• PyRIT Official Documentation
• Microsoft PyRIT GitHub

※ Garak Plugin names, PyRIT APIs, and Python requirements may vary by Version. Before executing commands, check the official documentation for the fixed release and verify with synthetic data in an authorized sandbox.