15-Stage Governance Pipeline

The Governance Pipeline is a 15-stage processing system that inspects, enforces policies, and audits every LLM request and response in real-time. All 15 stages execute in <200ms.

The 15 Stages

Input Pipeline (Stages 1-4)

Stage 1: Request Ingestion

Receive LLM request (prompt, model, parameters)
Extract metadata (user, application, timestamp)
Validate request format

Stage 2: PII Input Masking

Detect personally identifiable information (credit cards, SSNs, emails, phone numbers)
Mask or redact sensitive data
Log masking actions for audit

Stage 3: Injection Attack Detection

Detect prompt injection attempts
Identify jailbreaks and attempts to override instructions
Block malicious patterns

Stage 4: Policy Evaluation

Apply governance policies (Rego rules)
Check: allowed models, users, features, data classifications
Enforce compliance rules

Processing Pipeline (Stages 5-10)

Stage 5: Content Safety

Detect NSFW, harmful, illegal content
Screen for hateful language, violence, abuse
Apply content filtering policies

Stage 6: Truth Verification

Query Neural Cache for fact verification
Compare prompt against truth nuggets
Flag potential hallucinations before LLM call

Stage 7: Sensitive Data Classification

Classify request as: public, internal, restricted, classified
Tag with data residency requirements
Update classification metadata

Stage 8: LLM Call Routing

Route to appropriate LLM provider (OpenAI, Anthropic, Google, etc.)
Inject safety instructions based on policies
Add audit context to request

Stage 9: Response Retrieval

Receive LLM response
Extract response content, tokens used, model version
Capture metadata and timing

Stage 10: PII Output Masking

Detect PII in LLM response
Mask sensitive information
Log all masking for audit trail

Output Pipeline (Stages 11-15)

Stage 11: Response Fact-Check

Verify response against truth nuggets
Run NLI scoring (94%+ accuracy hallucination detection)
Flag inaccuracies

Stage 12: Response Safety

Check response for harmful content
Verify compliance with output policies
Ensure no data leakage

Stage 13: Correction Generation

If hallucination detected, generate correction
Create Neural Fact Sheet with sources
Prepare for auto-deployment

Stage 14: Audit Logging

Log entire request-response cycle
Hash-chain audit entries (SHA-256)
Write to immutable audit log

Stage 15: Response Return

Return response to application
Include metadata (fact-check status, corrections, policy violations)
Update metrics and dashboards

Stage Flow Diagram

Request
  ↓
[1] Ingestion
  ↓
[2] PII Input Masking
  ↓
[3] Injection Detection
  ↓
[4] Policy Evaluation ──→ BLOCK if policy violation
  ↓
[5] Content Safety ────→ FLAG if harmful content
  ↓
[6] Truth Verification ──→ WARN if hallucination likely
  ↓
[7] Data Classification
  ↓
[8] LLM Call Routing
  ↓
[9] Response Retrieval
  ↓
[10] PII Output Masking
  ↓
[11] Response Fact-Check ──→ ALERT if hallucination detected
  ↓
[12] Response Safety ────→ BLOCK if harmful output
  ↓
[13] Correction Generation
  ↓
[14] Audit Logging
  ↓
[15] Response Return
  ↓
Response

Execution Performance

Sub-200ms Guarantee

All 15 stages complete in <200ms:

Stage	Latency	Cumulative
1. Ingestion	0.5ms	0.5ms
2. PII Masking (input)	2ms	2.5ms
3. Injection Detection	1ms	3.5ms
4. Policy Evaluation	1ms	4.5ms
5. Content Safety	3ms	7.5ms
6. Truth Verification	8ms	15.5ms
7. Data Classification	0.5ms	16ms
8. LLM Routing	0.5ms	16.5ms
9. Response Retrieval	90ms	106.5ms
10. PII Masking (output)	2ms	108.5ms
11. Response Fact-Check	10ms	118.5ms
12. Response Safety	3ms	121.5ms
13. Correction Generation	20ms	141.5ms
14. Audit Logging	5ms	146.5ms
15. Response Return	1ms	147.5ms
Total	—	<150ms

Note: LLM call (Stage 9) is the dominant cost (90ms), not the governance processing.

Policy-Driven Decisions

Stages 4, 5, 12 enforce organizational policies written in Rego:

Example Policy

# Only allow ChatGPT and Claude
allow_models[model] {
    model == "gpt-4"
}
allow_models[model] {
    model == "claude-3-opus"
}

deny[msg] {
    not (input.model in allow_models)
    msg := sprintf("Model %v not allowed", [input.model])
}

# Don't allow PII in requests about healthcare
deny[msg] {
    input.classification == "healthcare"
    has_pii(input.prompt)
    msg := "Healthcare requests cannot contain PII"
}

# Fact-check marketing claims
require_factcheck[claim] {
    input.classification == "marketing"
    is_factual_claim(claim)
}

Audit Trail

Every request flows through the pipeline and creates an immutable audit record:

{
  "request_id": "req-abc123",
  "timestamp": "2024-01-15T10:30:45Z",
  "user_id": "user-456",
  "stages": [
    {
      "stage": 1,
      "name": "Ingestion",
      "duration_ms": 0.5,
      "status": "pass"
    },
    {
      "stage": 2,
      "name": "PII Input Masking",
      "duration_ms": 2,
      "status": "pass",
      "pii_masked": 1,
      "entities": ["email"]
    },
    {
      "stage": 4,
      "name": "Policy Evaluation",
      "duration_ms": 1,
      "status": "pass",
      "policies_evaluated": 3
    },
    {
      "stage": 11,
      "name": "Response Fact-Check",
      "duration_ms": 10,
      "status": "warning",
      "hallucinations_detected": 1,
      "nli_score": 0.42
    },
    {
      "stage": 13,
      "name": "Correction Generation",
      "duration_ms": 20,
      "status": "pass",
      "correction_generated": true
    },
    {
      "stage": 14,
      "name": "Audit Logging",
      "duration_ms": 5,
      "status": "pass",
      "hash": "sha256:abc...xyz"
    }
  ],
  "overall_status": "success",
  "total_duration_ms": 147.5,
  "response_received": true,
  "facts_verified": true,
  "corrections_needed": 1
}

Failure Modes

Fail-Open (Non-Blocking)

If any governance stage fails, system fails open (allows request through):

Stage 6 (Truth Verification) encounters error
  ↓
Log error and metrics
  ↓
Set hallucination_confidence = "unknown"
  ↓
Continue to Stage 7
  ↓
Return response with warning flag

This ensures pipeline failures don’t block user traffic.

Fail-Closed (Blocking)

Certain stages block requests on failure:

Stage 4 (Policy Evaluation): Block if policy says “deny”
Stage 5 (Content Safety): Block if harmful content detected
Stage 12 (Response Safety): Block if response violates safety rules

Fallback Behavior

Each stage has fallback logic:

Fallback Pattern:
  Try Stage Operation
    ├─ Success → Continue
    ├─ Recoverable Error → Use default conservative behavior
    └─ Unrecoverable Error → Fail open (log and continue)

Configuration

Configure pipeline behavior per application:

client.firewall.configure_pipeline(
    skip_stages=["pii_masking"],  # Disable PII masking
    stage_timeouts={
        "truth_verification": 20,  # 20ms timeout
        "response_factcheck": 15
    },
    fallback_behavior="fail_open",  # Default to allowing requests
    audit_log_detail="full"  # Log all details
)

Monitoring & Metrics

Dashboard shows pipeline health:

Navigate to Governance → Pipeline Health
View:
- Per-stage latency distribution
- Failure rates by stage
- Volume of requests at each stage
- Hallucinations detected per 1000 requests

metrics = client.firewall.get_pipeline_metrics(period="last_24_hours")

print(f"Total Requests: {metrics.request_count}")
print(f"Avg Pipeline Latency: {metrics.avg_latency_ms}ms")
print(f"Policy Violations: {metrics.policy_violations}")
print(f"Hallucinations Detected: {metrics.hallucinations_detected}")
print(f"Corrections Auto-Applied: {metrics.corrections_applied}")

Next Steps

Policies: Learn how to write Rego policies
Audit Trail: Understand immutable logging
Fact-Checking: Learn how hallucination detection works
Performance: Tune pipeline for your use case