Skip to content
TruthVouch Docs

Response Caching

TruthVouch automatically caches verification results to avoid redundant LLM calls. When a query has been processed before, the cached result is returned instantly instead of making another LLM request.

How It Works

When the same query is submitted again, TruthVouch returns the cached response in under 1ms. Cache entries expire based on configurable time windows (default: 1 hour for LLM responses, 24 hours for embeddings).

When no cached result exists, the query is sent to the LLM provider. The response is then stored in the cache for future lookups.

Cache keys are scoped per tenant — each client’s cached data is isolated and inaccessible to other clients.

Cache Policy Configuration

Clients on Professional tier and above can configure their cache behavior through the Settings > Cache tab:

  • Cache Duration (TTL): Choose how long verification results are cached before re-checking. Options range from 5 minutes to 7 days. Shorter TTLs provide fresher results but increase LLM costs; longer TTLs reduce costs but may serve slightly stale results.

  • Auto-refresh on Knowledge Updates: When enabled (the default), cached results are automatically cleared whenever you create, update, or delete truth nuggets. This ensures that verifications always reflect your latest knowledge base without manual intervention.

Cache Management

Admins can perform the following cache operations:

  • View Statistics: See the number of cached responses, cache hit rate (24-hour window), and estimated memory usage for your organization.

  • Clear All Cached Responses: Manually remove all cached verification results. New verifications will query the AI fresh, which may temporarily increase response times until the cache rebuilds.

  • Activity Log: View a record of recent cache operations, including when caches were cleared and by whom.

Cache Invalidation

Cache invalidation happens in two ways:

  1. Time-based expiry: Cache entries automatically expire after the configured TTL (default: 1 hour).
  2. Auto-invalidation: When truth nuggets are updated, the client’s cached results are cleared automatically (if the auto-refresh setting is enabled).

Monitoring

Platform administrators can view cache performance metrics including hit rate, memory usage, and per-client breakdown through the admin dashboard.

Next Steps