Logging, Transcripts and Audit

Purpose

TZ-12 adds a PostgreSQL audit trail for AI Operator calls. The audit layer records lifecycle events, state transitions, redacted transcripts, tool calls, KB searches, handoff/fallback outcomes, provider/media events, and retention runs.

Architecture

Audit is enabled by default with AUDIT_SINK=postgres. Runtime writes are best-effort by default: AUDIT_FAIL_CLOSED=false means a temporary audit write failure is logged in sanitized form and does not break the call.

The code is split into:

internal/audit: repository, service, export and retention logic.
internal/audit/redaction: deterministic redaction for text, JSON and byte payloads.
migrations/003_audit_tables.sql: audit schema.

Audit Tables

ai_calls: call-level record with masked caller number.
ai_call_events: lifecycle, state, tool, KB, handoff, provider, media and security events.
ai_transcript_events: redacted transcript events. Raw encrypted text is reserved and remains null by default.
ai_tool_audit: redacted tool args/results and allow/deny status.
ai_kb_audit: redacted query, result counts, citations and fallback flags.
ai_handoff_audit: handoff mode/status and redacted summary/target.
ai_provider_audit: provider counters and sanitized errors.
ai_media_audit: media stats only, never raw audio.
ai_audit_retention_runs: retention dry-run/confirm audit.

Event Taxonomy

Stable event names include call.started, call.ended, conversation.state_transition, conversation.denied_action, transcript.user.final, transcript.assistant.final, tool.call.completed, kb.search.completed, kb.search.no_answer, handoff.requested, fallback.triggered, provider.error, media.stats, and security.redaction_applied.

Transcript Policy

Defaults are privacy-first:

AUDIT_STORE_TRANSCRIPTS=true
AUDIT_STORE_TRANSCRIPT_DELTAS=false
AUDIT_STORE_RAW_TRANSCRIPTS=false
AUDIT_STORE_REDACTED_TRANSCRIPTS=true

Only redacted final transcript text is stored by default. Raw audio and base64 audio are never stored by the audit layer.

Redaction Rules

The redactor masks phone numbers, IIN-like 12 digit values, card-like 16 digit values, emails, OpenAI/API keys, Bearer tokens, password/secret/api_key values, PostgreSQL URL passwords, OTP/SMS codes in context, and account-like numbers. Recursive JSON redaction is used for tool args/results and metadata.

Retention Policy

Defaults:

audit events: 180 days
transcripts: 30 days
tool audit: 180 days
error audit: 365 days

audit-prune --dry-run is the default safe mode and records a retention run without deleting rows.

CLI

./bin/ai-operator audit-health --env /etc/ai-operator/ai-operator.env
./bin/ai-operator redaction-self-test --env /etc/ai-operator/ai-operator.env
./bin/ai-operator audit-self-test --env /etc/ai-operator/ai-operator.env
./bin/ai-operator audit-show-call --env /etc/ai-operator/ai-operator.env --call-id CALL_ID
./bin/ai-operator audit-export-call --env /etc/ai-operator/ai-operator.env --call-id CALL_ID --output /tmp/call-export.json
./bin/ai-operator audit-prune --env /etc/ai-operator/ai-operator.env --dry-run
./bin/ai-operator doctor --env /etc/ai-operator/ai-operator.env --check-audit

Export Format

Call export is JSON and contains the call row, events, redacted transcripts, tool audit, KB audit, handoff audit, provider audit and media audit. Raw transcript text and secrets are not included.

Privacy and Security

Do not enable raw transcript storage in production without a separate encryption and approval design. Do not log raw audio, base64 audio, full phone numbers, full IINs, full card numbers, database passwords, API keys, SIP credentials or ARI/AMI secrets.

Not Implemented Yet

Admin UI.
Encrypted raw transcript storage.
External SIEM integration.
Real call recordings.
Production route switch.

3.9 KiB Raw Permalink Blame History