ai-operator/docs/voice-provider.md

Voice Provider

Purpose

VoiceProvider is the replaceable voice AI boundary for the AI Operator. OpenAI is one adapter, not the foundation of the application.

Providers

• fake: offline provider for tests and local integration without external calls.
• openai_realtime: low-level WebSocket adapter for OpenAI Realtime.

OpenAI Realtime Flow

connect WebSocket
  -> session.created
  -> session.update
  -> input_audio_buffer.append
  -> response.output_audio.delta
  -> response.done / function_call

The adapter builds client events, parses server events, emits normalized VoiceEvent values, and never exposes the API key in logs.

Audio Path

Asterisk slin16 audio
  -> PCM16 validation
  -> optional 16000 <-> 24000 linear resampling
  -> base64 for OpenAI input_audio_buffer.append
  -> OpenAI output audio delta
  -> base64 decode
  -> optional resample back to Asterisk slin16
  -> Media Gateway SendAudio

Raw audio and base64 payloads are not logged or saved by default.

CLI Usage

OpenAI config check without network:

./bin/ai-operator doctor --env /etc/ai-operator/ai-operator.env --check-openai-config

Fake provider self-test:

./bin/ai-operator voice-provider-self-test --env /etc/ai-operator/ai-operator.env --provider fake

Live OpenAI smoke-test, only when OPENAI_API_KEY is configured:

./bin/ai-operator openai-smoke-test --env /etc/ai-operator/ai-operator.env

Test route run with fake provider:

./bin/ai-operator run --env /etc/ai-operator/ai-operator.env --ari-events --call-control --media --voice-provider fake

Test route run with OpenAI Realtime:

./bin/ai-operator run --env /etc/ai-operator/ai-operator.env --ari-events --call-control --media --voice-provider openai_realtime

Safety

• OpenAI API key is never printed.
• Authorization header is never printed.
• Raw audio is not logged or saved.
• Base64 audio is not logged.
• OpenAI is allowed only for explicit commands or --voice-provider openai_realtime.
• Production Kazakhtelecom routes remain disabled.

Cost Controls

Defaults for test mode:

• max session seconds: 60
• max input audio bytes: 10MB
• max output audio bytes: 10MB
• smoke-test does not call response.create unless explicitly requested

Limitations

• No language/region state machine yet.
• No RAG/knowledge base yet.
• No final agent prompt yet.
• Real tools are parsed but not executed.

TZ-06 Dialogue Prompt Integration

Voice providers receive state-aware prompt fragments from the Dialogue Orchestrator. The prompt is not trusted as the only guardrail; tool permission is enforced by Go policy before execution.

TZ-07 Transcript Integration

VoiceEventUserTranscriptDone can be passed to the Dialogue Orchestrator for language detection. Assistant transcript and audio events do not trigger language changes.

TZ-08 Region Resolver Update

TZ-08 adds an offline Kazakhstan Region Resolver with 20 enabled regions, stable region_code values, Almaty city/region clarification, false-positive protection, and explicit region-change policy. Production routing remains disabled; OpenAI, PostgreSQL, and Knowledge Base/RAG are still not used at this stage.

TZ-09 Knowledge Base Update

TZ-09 adds PostgreSQL + pgvector, KGA JSONL ingestion, fake embeddings, hybrid KB search, cross-language fallback for KK queries, citations, and search_knowledge_base integration behind the existing language/region guardrail. OpenAI is not called automatically and production routing remains disabled.

Tool Result Relay

VoiceProvider ToolCall events are routed through Call Manager to Dialogue Orchestrator. The resulting ToolResult is sent back to the provider with SendToolResult. FakeVoiceProvider records sent tool results for offline validation.

TZ-11 Handoff Events

VoiceProvider transcript events can trigger handoff detection. ToolCall events for request_human_handoff return structured ToolResults to the provider.

Audit Integration

Provider transcript finals and provider errors can be recorded through the audit service. Transcript deltas and raw transcripts are disabled by default.