Skip to content
TruthVouch Docs

Alert Workflows

Alert workflows manage the lifecycle of detected hallucinations: from creation through assignment, investigation, correction approval, deployment, and resolution. Automate routing, escalation, and SLA tracking.

Alert Lifecycle

1. Alert Created

Hallucination detected by Shield cross-check, anomaly detector, or PII scan.

What happens:

  • Alert assigned unique ID (alt_abc123…)
  • Severity calculated (Critical/High/Medium/Low)
  • Routed to appropriate channel (email, Slack, PagerDuty)
  • Assigned to default owner (or workflow rule determines owner)
  • Clock starts on SLA (1hr for Critical, 4hr for High, etc.)

Status: Open

2. Investigation

Team reviews alert and determines if it’s a true positive (real hallucination) or false positive.

Actions available:

  • Investigate — Open alert details to examine
    • View full AI response and Truth Nugget
    • See confidence score and evidence
    • Check if this AI engine has similar issues
    • Look at cross-checks against other AI engines
  • Mark as Reviewed — Confirm you’ve investigated
  • Request More Info — Ask someone else to gather details

Status: In Review (clock paused on SLA during investigation)

3. Decide on Action

Team decides: correct the AI’s outputs, update your Truth Nugget, or dismiss alert?

Three paths:

Path A: Approve Correction

  • AI hallucinated; we need to correct it
  • Correction method: Neural Fact Sheet or Direct Feedback
  • Team member approves correction
  • → Proceed to “Deploy Correction”

Path B: Edit Truth Nugget

  • Our Truth Nugget is wrong, not the AI
  • Click “Edit Nugget”
  • Update fact text, confidence, or source
  • Save
  • → Alert resolves automatically

Path C: Dismiss Alert

  • False positive (paraphrase OK, outdated nugget, marketing hyperbole)
  • Click “Dismiss”
  • Choose reason (dropdown)
  • System learns from dismissals
  • → Alert marked resolved

Status: Correction Approved, Awaiting Approval, or Dismissed

4. Deploy Correction (if applicable)

Correction method and deployment flow:

Neural Fact Sheet Method:

  • System generates AI-friendly factual summary (Neural Fact Sheet)
  • Fact sheet deployed to all monitored AI engines via RAG/knowledge base
  • Goal: AI learns correct fact and stops hallucinating

Deployment process:

  • Fact sheet created and versioned
  • Deployed to vector DB / knowledge base
  • All future queries against AI engines will see correct fact
  • Typically takes 15-30 seconds to deploy

Direct Feedback Method:

  • Explicit correction sent to AI engine (some APIs support this)
  • More direct but requires API support
  • Used alongside or instead of Neural Fact Sheet

Status: Deploying, Deployed

5. Verify Correction Effectiveness

Monitor if correction worked:

Verification process (automatic, runs 24-72 hours later):

  • Shield re-queries the same AI engine with same/similar prompt
  • If AI now gives correct answer → Correction verified
  • If AI still hallucinates → Correction ineffective (investigate why)

Outcomes:

  • Verified (95%+ of cases): Correction successful; alert resolved
  • Partially Verified: AI correct on similar queries, but not all variations
  • Unverified: Correction didn’t work; manual escalation needed

Status: Verifying → Verified or Escalated

6. Resolve

Alert closed after successful correction verification or dismissal.

Final statuses:

  • Resolved - Corrected — Hallucination corrected and verified
  • Resolved - Dismissed — False positive; correctly dismissed
  • Resolved - Nugget Updated — Our truth was wrong; updated
  • Escalated — Correction failed; requires manual investigation

Clock stops on SLA (response time recorded for reporting)

Assignment & Escalation

Automatic Assignment

Rules determine who owns the alert:

Example rules:

Rule 1: By Fact Category

  • Financial facts → Finance team
  • Product facts → Product team
  • Leadership facts → Communications team
  • Legal facts → Legal team

Rule 2: By AI Engine

  • ChatGPT issues → AI team
  • Custom model issues → Data Science team

Rule 3: By Severity

  • Critical → CEO/CTO
  • High → Department head
  • Medium → Manager
  • Low → Intern/junior person

Rule 4: By Time of Day

  • 9 AM - 6 PM → Day shift team member
  • After hours → On-call engineer

Manual Assignment

Alert owner can reassign to someone else:

  1. Click alert
  2. Click Assign to (or drag to team member)
  3. Notification sent to new owner
  4. Original owner removed (unless they add a comment)

Escalation

Alert can be escalated if owner doesn’t respond in time:

Escalation triggers:

  • No action taken within SLA → Auto-escalate
  • Owner manually escalates (clicks “Escalate”)
  • Issue affects multiple AI engines → Auto-escalate

Escalation path:

  • Alert owner → Their manager → Department head → C-suite

Example:

  • Level 1: Product team (24hr SLA) → No response after 4 hours
  • Level 2: Product manager (escalated to manager)
  • Level 3: VP of Product (no response after 8 hours)
  • Level 4: CEO/Board (critical issue affecting all systems)

SLA Tracking

Define SLAs

Set response time targets per severity:

SeverityResponse SLAResolution SLA
Critical15 min1 hour
High1 hour4 hours
Medium4 hours24 hours
Low24 hours1 week

Response SLA: Time to start investigating / assign to team

Resolution SLA: Time to resolve (correct deployed, false positive dismissed, or escalated)

SLA Compliance Dashboard

Metrics tracked:

  • % alerts responded to within SLA
  • % alerts resolved within SLA
  • Average response time (actual vs. target)
  • Average resolution time (actual vs. target)
  • Escalation rate (% of alerts requiring escalation)

Example metrics:

  • Critical: 95% responded within 15min, 88% resolved within 1hr
  • High: 92% responded within 1hr, 85% resolved within 4hr
  • Medium: 98% responded within 4hr, 80% resolved within 24hr
  • Low: 100% responded (not urgent), 70% resolved within 1 week

SLA Alerts

Alerts when SLA at risk:

  • At 50% of SLA → Warning notification
  • At 90% of SLA → Escalation escalates (or last-chance notification)
  • After SLA exceeded → Escalation (auto-escalate to manager)

Example: Critical alert created at 2:00 PM

  • 2:07 PM → 50% of 15-min SLA → “SLA at risk; you have 7.5 min left”
  • 2:13 PM → 90% of SLA → Auto-escalate to manager
  • 2:15 PM → SLA missed → Escalation to VP

Workflows by Alert Type

Hallucination Alerts

Typical workflow:

  1. Create alert (AI engine + fact + confidence)
  2. Assign to fact owner (Product for product facts, Finance for financial, etc.)
  3. Owner reviews (is this a real hallucination?)
  4. If yes → Approve neural fact sheet correction
  5. Deploy to vector DB
  6. Verify correction 24-72 hours later
  7. Resolve with resolution status

Average timeline: 4-24 hours (depends on verification)

PII Leakage Alerts

Typical workflow:

  1. Create alert (PII type + AI engine + response)
  2. Assign to Security team (high priority)
  3. Immediate action: Determine if actually PII or false positive
  4. If PII: Escalate to legal/compliance
  5. Notify affected parties (if required by GDPR/CCPA)
  6. Document for audit trail
  7. Correct: Update knowledge base to exclude PII

Average timeline: <1 hour (security-critical)

Compliance Violation Alerts

Typical workflow:

  1. Create alert (regulation + violation details)
  2. Assign to Compliance team
  3. Assess regulatory exposure and remediation options
  4. Correct: Update facts and policies
  5. Document: Create audit trail for regulators
  6. Verify: Re-check compliance

Average timeline: 1-3 days (regulatory considerations)

Bulk Actions

For multiple related alerts (e.g., same fact hallucinated 5 times):

Bulk Correction:

  1. Select multiple alerts (checkbox on each)
  2. Click Bulk Actions → Approve Correction
  3. One Neural Fact Sheet fixes all related alerts
  4. Deploy once; verification runs for all

Bulk Dismissal:

  1. Select multiple false positive alerts
  2. Click Bulk Actions → Dismiss All
  3. Choose reason
  4. All resolved at once

Bulk Assignment:

  1. Select multiple alerts
  2. Click Bulk Actions → Assign To
  3. Choose owner
  4. All reassigned at once

Workflow Automation

Create rules to automate common workflows:

Example 1: Auto-Correct Outdated Facts

IF: Alert severity = Low
  AND fact category = "Timeline" (e.g., "founded in...")
  AND confidence > 80%
THEN: Auto-approve correction (no human review)

Rationale: Low-severity timeline facts are safe to auto-correct.

Example 2: Auto-Escalate Compliance Issues

IF: Alert type = "Compliance Violation"
  AND alert not addressed within 30 minutes
THEN: Escalate to CEO + General Counsel

Rationale: Regulatory issues need executive attention immediately.

Example 3: Auto-Dismiss Marketing Hyperbole

IF: Alert about "best in class" claim
  AND fact category = "Product marketing"
THEN: Auto-dismiss with reason "Marketing hyperbole OK"

Rationale: Save team time on non-critical marketing claims.

Next Steps

  1. Define SLAs — Set response/resolution targets per severity
  2. Create assignment rules — Route by fact category, severity, team
  3. Set up escalation policy — Who escalates to whom?
  4. Configure notifications — Email/Slack/PagerDuty for each level
  5. Run pilot — Try workflow with one team for 1 week
  6. Optimize based on metrics — Adjust SLAs and routing rules