Writing Policies

Policy Basics

A policy defines a rule that all AI requests/responses must follow automatically. If violated, TruthVouch blocks or warns.

Simple Policy Example:

Name: Block API Keys in Requests
Applies To: All AI Models
Condition: Request contains pattern "api_key = ..."
Action: Block
Message: "API keys must not be sent to AI models"

Policy Structure

Via UI (Visual Policy Builder)

1. Go to Governance → Policies → + New Policy
2. Fill form:
   • Name: Human-readable identifier
   • Description: What this policy does
   • Applies To: All models, specific model, or by user
   • Type: Visual rule or Rego code
   • Condition: What to check
   • Action: Block, warn, or log
   • Message: User-facing error message
3. Click Test
4. Click Deploy

Via Code (Rego Policies)

Use Open Policy Agent (OPA) Rego language for complex logic:

package policies.security

# Policy: Block PII in prompts
deny[msg] {
    input.type == "request"
    contains_pii(input.text)
    msg := "Requests cannot contain PII; please remove before submitting"
}

contains_pii(text) {
    re_match("\\d{3}-\\d{2}-\\d{4}", text)  # SSN
}

Visual Policy Types

Pattern Matching

Check if request/response matches a pattern:

Condition: Text contains regex pattern
Pattern: \d{4}-\d{4}-\d{4}-\d{4}  (credit cards)
Action: Block

Allowlist

Only allow items in a list:

Condition: Model is in list
List: ["gpt-4", "gpt-4-turbo", "claude-3-opus"]
Action: Block (if not in list)

Blocklist

Block items in a list:

Condition: User is in list
List: ["test-user@company.com"]
Action: Block (if in list)

Threshold

Check if a value exceeds/falls below threshold:

Condition: Tokens used > value
Threshold: 10000
Action: Block

Rego Policies

Use Rego for complex business logic:

Example 1: Token Budget Per Department

package policies.token_budget

deny[msg] {
    input.type == "request"
    department := data.departments[input.user_id]
    daily_tokens := data.token_usage[department]["today"]
    daily_tokens + input.tokens > data.limits[department]
    msg := sprintf(
        "Department %v daily token limit exceeded (%v remaining)",
        [department, data.limits[department] - daily_tokens]
    )
}

Example 2: Model Approval by Role

package policies.model_approval

deny[msg] {
    input.type == "request"
    model := input.model
    user_role := data.users[input.user_id].role
    not user_can_use_model(user_role, model)
    msg := sprintf("Your role (%v) cannot use %v", [user_role, model])
}

user_can_use_model(role, model) {
    allowed_models := data.role_permissions[role].models
    model in allowed_models
}

Example 3: Content Safety for Outputs

package policies.content_safety

deny[msg] {
    input.type == "response"
    input.safety_flags.toxicity > 0.7
    input.user_type == "external"  # Only for external users
    msg := "Response contains toxic content"
}

Policy Scope

By Model

Apply policy only to specific models:

Applies To: gpt-4 only

By User/Role

Apply only to certain users:

Applies To: Users with role "analyst"

By Department

Apply only to departments:

Applies To: HR department

All (Default)

Apply to every request:

Applies To: All users, all models

Actions

Block

Stops the request/response from proceeding.

Action: Block
Message: "Policy violation: PII detected"

User sees error and request doesn’t reach AI.

Warn

Allows request through but logs warning.

Action: Warn
Message: "Warning: Request may violate data policy"
Severity: Medium

Request continues, but audit trail shows warning.

Log

Records policy evaluation without user notification.

Action: Log
Severity: Low

Useful for monitoring policies before deployment.

Real-World Policy Examples

Policy: Block Unmasked PII

Scenario: Your compliance officer requires all PII to be masked before sending to external AI.

package policies.pii_protection

deny[msg] {
    input.type == "request"
    input.destination == "external"
    contains_unmasked_pii(input.text)
    msg := "Cannot send unmasked PII to external AI services"
}

contains_unmasked_pii(text) {
    re_match("\\d{3}-\\d{2}-\\d{4}", text)  # SSN pattern
}

Policy: Rate Limit by Department

Scenario: Marketing gets 100K tokens/month, Engineering gets 500K.

package policies.department_tokens

deny[msg] {
    user := input.user_id
    dept := data.user_departments[user]
    limit := data.dept_limits[dept]
    used := data.monthly_tokens[dept]
    used + input.tokens > limit
    msg := sprintf("Department token limit exceeded")
}

Policy: Model Whitelist

Scenario: Only approved models allowed.

package policies.model_approval

deny[msg] {
    model := input.model
    approved := ["gpt-4", "claude-3-opus", "claude-3-sonnet"]
    not model in approved
    msg := sprintf("Model %v is not approved", [model])
}

Testing Before Deployment

Critical: Always test policies before going live.

1. Click Test Policy button
2. Enter test cases:

   Test 1: SSN pattern "123-45-6789"
   Expected: BLOCKED
   
   Test 2: Normal text "What is AI?"
   Expected: ALLOWED
   
   Test 3: Partial SSN "123-45"
   Expected: ALLOWED

3. Run tests
4. Verify results match expectations
5. Only then click Deploy

Deployment Strategies

Immediate Deployment

Deploy to all users immediately:

1. Click Deploy
2. Select "Immediate"
3. Policy goes live now

When to use: Low-risk policies (informational), critical security policies.

Staged Rollout

Deploy to subset of users first:

1. Click Deploy
2. Select "Staged"
3. Day 1: 10% of users
4. Day 2: 25% of users
5. Day 3: 50% of users
6. Day 4: 100% of users

When to use: Major policy changes, need to monitor impact.

Shadow Mode

Log violations without blocking:

1. Deploy with Action: "Log"
2. Monitor violations for 1 week
3. If no issues, change to Block

When to use: Uncertain about policy impact, want data before enforcing.

Policy Approval Workflow

For governance, require approval before deployment:

1. Policy Creator writes policy and tests
2. Click “Request Approval”
3. Approval Notified (e.g., security team)
4. Reviewer examines policy, tests, impact
5. Reviewer approves or rejects with feedback
6. If Approved: Policy can now be deployed
7. If Rejected: Creator can revise and resubmit

Setup Approval Requirements:

1. Go to Settings → Governance → Approval Workflow
2. Require Approval For: Policy changes
3. Approval Required From: Select roles/users
4. Auto-Approve After: 48 hours (optional timeout)

Monitoring Policies

After deployment:

1. Go to Governance → Reports → Policies
2. See Violations chart:
   • Which policies triggered most
   • Trend over time
3. See False Positives:
   • Legitimate requests incorrectly blocked
4. Adjust thresholds if needed

Retiring Policies

When a policy is no longer needed:

1. Click Disable to stop enforcement
2. Policy still visible in history
3. Violations still logged for audit trail
4. Later: Click Archive to move out of active list

API Access

Create/update policies programmatically:

curl -X POST http://localhost:5000/api/v1/governance/policies \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "name": "Block PII",
    "description": "Prevent PII in requests",
    "type": "rego",
    "code": "package policies.pii\ndeny[msg] { ... }",
    "action": "block",
    "applies_to": "all"
  }'

Best Practices

1. Start Simple: Regex policies before Rego
2. Test Thoroughly: Use test mode extensively
3. Monitor Impact: Watch false positives
4. Document Reason: Add comment explaining why
5. Version Control: Track changes in git
6. Regular Review: Check if still relevant quarterly

Related Topics

• Policy-as-Code (Rego)
• Testing Policies
• Policy Approvals
• Developer Rules