Deploy: SaaS

Overview

The SaaS deployment model hosts the Firewall as a managed service in TruthVouch’s infrastructure. You redirect API calls through our secure proxy endpoint, and we handle all scanning, scaling, and monitoring. This is the fastest way to get started.

Architecture

Your App
    |
    v
TruthVouch Proxy (firewall.truthvouch.io)
    |
    +---> Rate Limiter, PII Masking, Injection Detection
    |
    +---> AI Provider (OpenAI, Anthropic, etc.)
    |
    v
Your App (receives scanned response)

Setup (5 Minutes)

Step 1: Generate API Key

1. Log in to TruthVouch Dashboard
2. Go to Governance → Settings → API Keys
3. Click + New API Key
4. Name it (e.g., “Firewall Proxy”)
5. Grant permission: “Firewall”
6. Copy the key (looks like tvk_sk_...)

Step 2: Update Your SDK Calls

Replace your direct AI provider calls with TruthVouch proxy calls.

OpenAI Example (Before):

import openai

openai.api_key = "sk-..."
response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "What is AI?"}]
)

OpenAI Example (After):

import openai

openai.api_key = "tvk_sk_..."  # TruthVouch API key
openai.api_base = "https://firewall.truthvouch.io/openai/v1"  # New proxy endpoint
response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "What is AI?"}]
)

Anthropic Example (Before):

import anthropic

client = anthropic.Anthropic(api_key="sk-ant-...")
response = client.messages.create(
    model="claude-3-sonnet",
    messages=[{"role": "user", "content": "What is AI?"}]
)

Anthropic Example (After):

import anthropic

client = anthropic.Anthropic(
    api_key="tvk_sk_...",  # TruthVouch API key
    base_url="https://firewall.truthvouch.io/anthropic"
)
response = client.messages.create(
    model="claude-3-sonnet",
    messages=[{"role": "user", "content": "What is AI?"}]
)

REST API Example:

# Before: Direct to OpenAI
curl https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -d '{"model": "gpt-4", "messages": [...]}'

# After: Through TruthVouch
curl https://firewall.truthvouch.io/openai/v1/chat/completions \
  -H "Authorization: Bearer tvk_sk_..." \
  -d '{"model": "gpt-4", "messages": [...]}'

Step 3: Configure Firewall Policies

1. Go to Governance → Firewall → Configuration
2. Choose your preset (High Security, Standard, High Throughput) or customize
3. Click Deploy

That’s it! Requests now flow through the Firewall.

Supported Providers

• OpenAI (GPT-4, GPT-3.5, fine-tunes)
• Anthropic (Claude 3 family)
• Google (Gemini)
• Cohere (Command, Embed)
• Ollama (self-hosted models)
• Azure OpenAI
• LiteLLM (multi-provider)

Proxy Endpoints

Provider	Endpoint
OpenAI	https://firewall.truthvouch.io/openai/v1
Anthropic	https://firewall.truthvouch.io/anthropic
Google	https://firewall.truthvouch.io/google
Cohere	https://firewall.truthvouch.io/cohere
Azure OpenAI	https://firewall.truthvouch.io/azure/<your-deployment>

Monitoring & Debugging

View Request Logs

1. Go to Governance → Audit → Firewall Requests
2. Filter by date, status, user, or scan stage
3. Click on a request to see details (input, output, which stages fired)

Debugging Failed Requests

If a request was blocked:

1. Find it in the audit log
2. Click to expand
3. See which stage blocked it and why
4. Check the violation details (e.g., “PII detected: email_address”)
5. Add to allowlist if it’s a false positive

Check Performance

Go to Governance → Reports → Firewall Performance:

• P50/P95/P99 Latency: Time added to each request
• Throughput: Requests/second processed
• Error Rate: Timeouts, configuration errors
• Stage Duration: Which stages are slowest

Latency & Performance

Typical Latency

• SaaS Proxy: 50-150ms added per request
  • Network round-trip: 30-80ms
  • Scanning stages: 20-70ms
  • Based on your geography and traffic patterns

Optimizing for Speed

1. Use Regional Endpoint: If available, specify your region (e.g., firewall.eu.truthvouch.io)
2. Batch Requests: Send multiple prompts in one batch for better throughput
3. Cache Responses: Store frequent queries to skip firewall on repeats
4. Async Scanning: Enable async mode for non-critical requests
5. Disable Unused Stages: Turn off PII masking if you don’t need it

SLA

• 99.9% Uptime: TruthVouch maintains 99.9% availability (26 minutes/month downtime)
• Max Latency: 95th percentile <= 200ms under normal load
• Redundancy: Multi-region failover, automatic retry

Security & Privacy

Encryption

• In Transit: TLS 1.3 (HTTPS), Perfect Forward Secrecy
• At Rest: AES-256 encryption of audit logs
• API Keys: Hashed using bcrypt, never stored in logs

Data Retention

• Audit Logs: 90 days by default (configurable up to 2 years)
• Cached Embeddings: 30 days (used for similarity scoring)
• Conversation State: 24 hours (for multi-turn context)
• Deleted Immediately: Actual AI responses after processing

Compliance

• SOC 2 Type II: Audited and certified
• GDPR: Data processing agreement (DPA) available
• HIPAA: BAA available for regulated environments
• FedRAMP: In progress

Quotas & Pricing

Standard Tier

• Requests: 100K/month included
• Concurrent: 10 simultaneous requests
• Stages: All 15 available
• Cost: $29/month + $0.01 per 1K requests over quota

Enterprise Tier

• Requests: Unlimited
• Concurrent: 1000+ simultaneous requests
• Stages: All + custom stages
• Support: Priority support, dedicated account manager
• Cost: Custom pricing

Check Settings → Billing for your current usage and limits.

Advanced Configuration

Custom Headers

Pass metadata to the Firewall:

curl https://firewall.truthvouch.io/openai/v1/chat/completions \
  -H "Authorization: Bearer tvk_sk_..." \
  -H "X-TruthVouch-User-ID: user123" \
  -H "X-TruthVouch-Session-ID: sess456" \
  -d '{"model": "gpt-4", ...}'

Webhook Notifications

Get notified when security violations occur:

1. Go to Governance → Settings → Webhooks
2. Click + Add Webhook
3. Enter your endpoint (e.g., https://yourapp.com/security-alert)
4. Select which events trigger it (e.g., “Injection detected”, “PII in output”)
5. Save

TruthVouch will POST to your endpoint with violation details.

Rate Limiting Per User

Override default rate limits for specific users:

curl -X POST http://localhost:5000/api/v1/governance/rate-limits \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "user_id": "user@company.com",
    "requests_per_minute": 500
  }'

Troubleshooting

”401 Unauthorized”

Check that your API key is correct and hasn’t expired. Generate a new one in the dashboard.

”429 Too Many Requests”

You’ve exceeded rate limits. Check your quota or request limit increase.

”502 Bad Gateway”

TruthVouch proxy had an error. Requests are retried automatically. If it persists, contact support.

High Latency

Check Governance → Reports → see which stage is slow. Disable unused stages or increase timeout thresholds.

False Positives (Legitimate Content Blocked)

Add to allowlist: Governance → Firewall → Allowlists → add pattern.

Migrating from Self-Hosted

1. Set up SaaS proxy endpoint (5 min above)
2. Point a % of traffic to new proxy for testing
3. Monitor audit logs to ensure same behavior
4. Gradually increase % to 100%
5. Decommission self-hosted Firewall

Related Topics

• Firewall Overview
• Firewall Configuration
• Performance Tuning
• Self-Hosted Deployment