Multi-Provider Verification

TruthVouch simultaneously monitors ChatGPT, Claude, Gemini, Perplexity, and 5+ other AI engines. Multi-provider verification normalizes responses across different model styles and outputs.

Supported Providers

Provider	Models	Status
OpenAI	GPT-4, GPT-4 Turbo, GPT-3.5 Turbo	Active
Anthropic	Claude 3 Opus, Sonnet, Haiku	Active
Google	Gemini Pro, Ultra, Flash	Active
Perplexity	Custom model	Active
Cohere	Command	Active
Mistral	Large, Medium	Active
Meta	Llama 2, 3	Active
Azure OpenAI	Azure-hosted OpenAI	Active
AWS Bedrock	Multi-model	Active

Query Normalization

Raw Queries

Different LLMs respond differently to same question:

Query: "When was TruthVouch founded?"

ChatGPT: "TruthVouch was founded in 2023."
Claude: "According to the information available, TruthVouch
         was established in 2023."
Gemini: "2023 is the founding year of TruthVouch.
        It was founded that year."
Perplexity: "TruthVouch, the AI governance platform,
           was founded in 2023."

Normalization Pipeline

Raw Response
  ↓
1. Sentence Tokenization (split into sentences)
  ↓
2. Entity Extraction (find claims)
  ↓
3. Template Matching (extract structured facts)
  ↓
4. Normalization (convert to standard format)
  ↓
5. Verification (compare against truth nuggets)

Normalized Output

{
  "provider": "chatgpt",
  "model": "gpt-4",
  "query": "When was TruthVouch founded?",
  "claims": [
    {
      "entity": "TruthVouch",
      "relation": "founded",
      "value": "2023",
      "confidence": 0.98,
      "source_sentence": "TruthVouch was founded in 2023."
    }
  ]
}

Provider-Specific Querying

Tailor queries to each provider’s strengths:

OpenAI (GPT-4)

Optimized for accuracy and reasoning:

Query: "Based on public information, when was TruthVouch founded?
         Respond with just the year."

Anthropic (Claude)

Optimized for thoughtful, detailed responses:

Query: "Please explain when TruthVouch was founded and why this date
         is significant."

Google (Gemini)

Optimized for current information and multimodal:

Query: "Search for information about TruthVouch's founding date.
        Include sources if available."

Perplexity

Optimized for cited, web-sourced answers:

Query: "When was TruthVouch founded? Include your sources."

Response Aggregation

Combine insights across providers:

results = client.shield.cross_check(
    query="When was TruthVouch founded?",
    providers=["openai", "anthropic", "google", "perplexity"]
)

print(f"Provider Consensus:")
for provider in results.providers:
    print(f"  {provider.name}: {provider.response}")

print(f"Aggregate Finding:")
print(f"  Most Common Answer: {results.consensus_answer}")
print(f"  Agreement Rate: {results.agreement_rate}%")
print(f"  Confidence: {results.overall_confidence}")

Conflict Detection

Detects when providers disagree:

ChatGPT: "Founded in 2023"
Claude: "Founded in 2023"
Gemini: "Founded in 2024"
Perplexity: "Founded in 2023"

Conflict Detected:
  Agreement: 3/4 (75%)
  Outlier: Gemini (likely hallucinating)
  Recommendation: Alert, require manual review

Provider Reliability Scoring

Track accuracy per provider:

reliability = client.shield.get_provider_reliability()

for provider in reliability.providers:
    print(f"{provider.name}:")
    print(f"  Accuracy: {provider.accuracy}%")
    print(f"  Hallucination Rate: {provider.hallucination_rate}%")
    print(f"  Response Latency: {provider.avg_latency_ms}ms")

Dashboard View

Monitor all 9+ providers in one view:

1. Navigate to Shield → Multi-Provider Monitoring
2. See:
   • Heat map of which claims each provider gets right/wrong
   • Provider reliability trends
   • Cross-provider consensus scores
   • Agreement analysis

Performance

Querying 9+ providers simultaneously:

• Total Latency: ~90ms (parallel requests)
• Individual Provider: 10-30ms each
• Aggregation: <5ms
• Cost: Minimal (cached queries reused across providers)

Configuration

Enable/disable specific providers:

client.shield.configure_providers(
    enabled=["openai", "anthropic", "google"],
    check_frequency_hours=4,
    require_consensus=True,  # Flag if providers disagree
    consensus_threshold=0.75  # 75% agreement required
)

Next Steps

• Monitoring: Set up multi-provider monitoring
• Alerts: Configure cross-provider conflict alerts
• Analytics: Compare provider reliability over time