call-center/docs/architecture/voice-ai-v1.md

Voice AI V1 Technical Design

Status: Proposed

1. Goal

Add an AI-first voice operator on top of the accepted Asterisk voice baseline without replacing the current telephony stack.

Voice AI V1 must:

• answer selected inbound calls before a human joins;
• reuse the existing interaction, customer, KB, queue, and operator UI model;
• keep Asterisk as the system of record for call lifecycle, transfer, recording, SIP/WebRTC, and queue ownership;
• store business state in platform tables, not only inside model context;
• support deterministic AI-to-human handoff into the existing human queue.

2. Non-goals

Voice AI V1 does not:

• replace Asterisk, AMI, dialplan, queueing, SIP/WebRTC, or recording;
• introduce a separate AI telephony platform or second source of truth for calls;
• push full CRM history, full call transcript, or raw recordings into every model turn;
• introduce a multi-agent swarm;
• add attended transfer, conference, or full human-to-AI return in the same live call.

return-to-ai after human takeover is intentionally deferred to V2. It requires a second telephony redirect and fresh media re-attachment, which is riskier than AI-first plus human handoff.

3. Existing Baseline We Build On

The current repo already has the pieces needed for Voice AI V1:

• asterisk_bridge_service owns AMI ingestion, live call tracking, claim/hangup/transfer, and recording import.
• voice_adapter_service stores telephony lifecycle events.
• interaction_service owns the canonical interaction lifecycle.
• customer_service plus customer_external_identities already provide cross-channel identity linking.
• ai_orchestrator_service already implements the Telegram AI pattern:
  • deterministic orchestration;
  • explicit AI disclosure;
  • ai_sessions and ai_turns;
  • AI-to-human handoff;
  • source-of-truth separation between channel service and AI service.
• /operator already has a hybrid Telegram AI UX with AI badges and AI summary cards.

Voice AI V1 should copy the Telegram AI service boundary pattern, not the Telegram channel model itself.

4. Target Architecture

4.1 Service responsibilities

Service	Role in Voice AI V1
asterisk_bridge_service	Keeps telephony truth, selects AI path for eligible calls, starts/stops AI runtime sessions, performs AI-to-human redirect, exposes operator-facing voice AI summary.
ai_voice_runtime_service	New realtime service. Owns media session state, ASR/TTS streaming, turn detection, barge-in, runtime latency budget, and handoff requests.
ai_orchestrator_service	Reused and extended. Owns deterministic voice decisioning, filtered context assembly, KB/tool usage, safe business policies, and AI summary generation.
interaction_service	Remains the owner of interaction status and timeline; gains an internal timeline append endpoint for additive AI events.
voice_adapter_service	Stays the owner of telephony event persistence only. It does not become the AI runtime.
ui/operator	Reuses the current live call and popup flow; adds voice AI badges and handoff summary, but no separate AI telephony workspace.

4.2 High-level component view

flowchart LR
    A["Asterisk<br/>queue, redirect, recording, SIP/WebRTC"] --> B["asterisk_bridge_service<br/>call truth + control plane"]
    B --> C["ai_voice_runtime_service<br/>media bridge + ASR/TTS + barge-in"]
    C --> D["ai_orchestrator_service<br/>policy + tools + summaries"]
    D --> E["customer_service / shared DB<br/>customer profile + memory"]
    D --> F["interaction_service<br/>interaction status + timeline"]
    D --> G["kb_service<br/>KB lookup"]
    B --> H["operator UI<br/>live call popup + voice AI summary"]

4.3 Source-of-truth rule

• Telephony truth stays in AsteriskCallLinkRow, AMI events, queue redirect, and recording flow.
• AI truth stays in voice_ai_sessions, ai_sessions, ai_turns, and transcript rows.
• Interaction truth stays in Interaction plus InteractionTimeline.
• Customer truth stays in Customer plus CustomerExternalIdentity.

No single prompt becomes the long-term state container.

5. Call Flow

5.1 AI-first inbound voice flow

sequenceDiagram
    participant A as Asterisk
    participant B as asterisk_bridge_service
    participant R as ai_voice_runtime_service
    participant O as ai_orchestrator_service
    participant I as interaction_service

    A->>B: MVPCCCallStarted
    B->>I: create / reuse interaction
    B->>B: create or update AsteriskCallLinkRow
    B->>R: POST /internal/voice-ai/sessions
    R->>O: POST /ai/voice/sessions/{id}/start
    R->>A: play AI disclosure + greeting
    A->>R: caller audio stream
    R->>O: POST /ai/voice/sessions/{id}/turns
    O-->>R: reply plan or handoff decision
    R->>A: TTS playback

Detailed steps:

1. asterisk_bridge_service receives the existing MVPCCCallStarted event.
2. It keeps the current behavior of creating or resolving interaction_id and AsteriskCallLinkRow.
3. It evaluates whether this queue or IVR outcome should go to AI-first mode.
4. If not AI-eligible, the current human flow stays unchanged.
5. If AI-eligible:
   • create voice_ai_session;
   • create or link generic ai_session with channel="voice";
   • mark the call row as AI-owned;
   • start a runtime control session in ai_voice_runtime_service;
   • attach the customer leg to the AI media bridge.
6. ai_voice_runtime_service plays a short disclosure and greeting.
7. Finalized caller utterances are sent to ai_orchestrator_service.
8. The orchestrator loads filtered business context, runs deterministic policy and KB/tool lookup, and returns a structured decision.
9. The runtime either:
   • plays TTS back to the caller; or
   • requests handoff to a human queue.

5.2 AI-to-human handoff

sequenceDiagram
    participant R as ai_voice_runtime_service
    participant O as ai_orchestrator_service
    participant B as asterisk_bridge_service
    participant A as Asterisk
    participant U as Operator UI

    R->>O: POST voice turn
    O-->>R: needs_handoff=true + reason + summary
    R->>B: POST /internal/voice-ai/calls/{call_id}/handoff
    B->>B: mark ai_state=handoff_required
    B->>A: redirect caller leg to existing human queue
    A->>U: normal operator incoming call path
    U->>B: GET /asterisk/live-calls/{call_id}/ai-summary
    B-->>U: AI handoff summary

Handoff rules:

• handoff happens only through the existing Asterisk queue path;
• the AI runtime never calls an operator directly and never becomes a second queueing owner;
• interaction_id stays the same through AI and human phases;
• the operator receives a concise AI summary before or during claim.

5.3 Call end

• MVPCCCallEnded and MVPCCRecordingReady continue to come from Asterisk through the existing bridge.
• The bridge informs the runtime session that telephony ended.
• The runtime closes voice_ai_session.
• The orchestrator writes the final AI summary and marks ai_session closed.
• The interaction timeline gets final AI events if they were not written yet.

6. Queue Selection and AI Eligibility

Voice AI V1 should start with config-driven AI eligibility, not a new admin UI.

Recommended V1 config:

AI_VOICE_QUEUE_CONFIG_JSON

Example shape:

{
  "voice_lab": {
    "mode": "ai_first",
    "agent_profile": "voice_support",
    "handoff_queue_code": "voice_lab",
    "language": "ru"
  }
}

Why config-first:

• it avoids touching the accepted routing UI and queue schema in the first step;
• it keeps rollout reversible per queue;
• it matches the current repo style where telephony behavior is still largely env-driven.

The longer-term path can move this policy into routing/admin later.

7. Data Model

7.1 Reused tables

These tables stay authoritative and should be reused:

• interactions
• interaction_timelines
• customer_external_identities
• asterisk_call_links
• voice_events
• call_recordings
• ai_sessions
• ai_turns

Voice customer linking should reuse customer_external_identities with:

• channel = "voice"
• external_subject = normalized caller number

No new identity table is needed.

7.2 New table: voice_ai_sessions

This table stores runtime state that does not fit cleanly into generic ai_sessions.

Suggested columns:

Column	Type	Purpose
id	integer pk	Internal row id
session_id	string unique	Public runtime session id
call_id	string unique index	Current Asterisk customer-leg call id
linked_id	string index	Asterisk linked id for recovery if call id changes
interaction_id	string index	Shared interaction
customer_id	string index nullable	Linked customer
queue_id	string index	Original platform queue
ai_session_id	string index	Generic AI session link
agent_profile	string index	Voice policy profile
status	string index	queued, greeting, listening, thinking, speaking, handoff_requested, human_owned, completed, error
language	string index nullable	Current active language
asr_provider	string nullable	Selected ASR backend
tts_provider	string nullable	Selected TTS backend
handoff_reason	text nullable	Last handoff reason
handoff_target_queue_id	string nullable	Human target queue
disclosure_played_at	string nullable	When AI disclosure was first spoken
last_user_utterance_at	string nullable	Latest finalized caller speech
last_ai_reply_at	string nullable	Latest completed AI reply
started_at	string index	Session start
updated_at	string index	Last state update
ended_at	string index nullable	Session end

7.3 New table: voice_transcript_segments

This table stores finalized voice transcript units separately from generic AI turns.

Suggested columns:

Column	Type	Purpose
id	integer pk	Internal row id
segment_id	string unique	Public segment id
session_id	string index	voice_ai_sessions.session_id
call_id	string index	Voice call correlation
interaction_id	string index	Interaction correlation
speaker	string index	caller, assistant, system, operator
source_type	string index	asr, tts, handoff_summary, system
sequence_no	integer index	Ordered segment number
text	text	Final transcript text
confidence	number nullable	ASR confidence when applicable
is_final	boolean	Finalized transcript only in V1, but keep the flag for future partials
barge_in_interrupted	boolean	Whether the assistant segment was interrupted
payload_json	text	Provider metadata, timestamps, tool refs
created_at	string index	Write time

V1 should store finalized transcript segments only. Partial ASR events can stay in memory inside the runtime.

7.4 Additive columns on asterisk_call_links

This mirrors the Telegram pattern where the channel source-of-truth row also exposes AI status.

Add:

• voice_session_id VARCHAR(64) NULL
• ai_session_id VARCHAR(64) NULL
• ai_state VARCHAR(32) NULL
• ai_handoff_reason TEXT NULL
• ai_last_model_at VARCHAR(64) NULL

Suggested ai_state values:

• queued
• greeting
• listening
• thinking
• active
• handoff_required
• human_owned
• closed
• error

This lets /asterisk/live-calls and /asterisk/recent-calls drive UI chips directly without extra joins on every poll.

7.5 Additive column on ai_sessions

Add:

• call_id VARCHAR(128) NULL

Reason:

• Telegram already uses thread_id;
• voice needs a direct telephony key for fast lookup and summary generation;
• this keeps ai_sessions truly cross-channel instead of Telegram-shaped.

7.6 Reuse of ai_turns

Reuse ai_turns for:

• finalized user turn seen by the orchestrator;
• model reply plan;
• tool invocation results;
• summary turn written during handoff or closure.

Do not reuse ai_jobs for voice V1. Telegram jobs are thread-triggered and synchronous voice turn processing does not need that queue model.

8. API Design

8.1 asterisk_bridge_service -> ai_voice_runtime_service

New internal control-plane endpoints:

POST /internal/voice-ai/sessions

Starts a runtime session for an already accepted telephony call.

Request:

{
  "call_id": "1740912000.12",
  "linked_id": "1740912000.12",
  "interaction_id": "int_...",
  "queue_id": "que_...",
  "caller_number": "+7701...",
  "caller_name": "Lab Caller",
  "agent_profile": "voice_support",
  "language_hint": "ru",
  "handoff_queue_id": "que_...",
  "metadata": {
    "queue_code": "voice_lab",
    "direction": "inbound"
  }
}

Response:

{
  "voice_session_id": "avs_...",
  "ai_session_id": "ais_...",
  "status": "queued"
}

POST /internal/voice-ai/sessions/{session_id}/telephony-events

Bridge notifies runtime about:

• call.connected
• call.ended
• recording.ready
• operator.connected

This keeps telephony truth in the bridge while runtime stays current.

8.2 ai_voice_runtime_service -> ai_orchestrator_service

POST /ai/voice/sessions/{session_id}/start

Creates or reopens the generic AI session and returns greeting policy:

{
  "voice_session_id": "avs_...",
  "call_id": "1740912000.12",
  "interaction_id": "int_...",
  "customer_id": "cus_...",
  "language_hint": "ru",
  "agent_profile": "voice_support"
}

Response:

{
  "session_id": "ais_...",
  "language": "ru",
  "greeting_text": "Здравствуйте. Я AI-оператор компании...",
  "disclosure_required": true
}

POST /ai/voice/sessions/{session_id}/turns

Main deterministic decision endpoint.

Request:

{
  "voice_session_id": "avs_...",
  "call_id": "1740912000.12",
  "interaction_id": "int_...",
  "transcript_text": "Хочу узнать статус заявки",
  "language": "ru",
  "sequence_no": 3,
  "barge_in": false,
  "metadata": {
    "turn_duration_ms": 4200
  }
}

Response:

{
  "language": "ru",
  "intent": "status_check",
  "reply_text": "Я AI-оператор компании. Проверяю данные по обращению...",
  "confidence": 0.83,
  "needs_handoff": false,
  "handoff_reason": null,
  "case_action": "keep_open",
  "kb_refs": ["art_..."],
  "summary_text": "Клиент уточняет статус заявки.",
  "model": "gpt-4o-mini",
  "latency_ms": 780
}

case_action should stay aligned with the existing Telegram pattern:

• none
• keep_open
• close
• escalate

POST /ai/voice/sessions/{session_id}/close

Finalizes AI state and writes the terminal summary.

8.3 ai_voice_runtime_service -> asterisk_bridge_service

POST /internal/voice-ai/calls/{call_id}/handoff

Requests redirect of the current customer leg into the existing human queue.

Request:

{
  "voice_session_id": "avs_...",
  "ai_session_id": "ais_...",
  "interaction_id": "int_...",
  "target_queue_id": "que_...",
  "reason": "Нужен человек для чувствительного запроса.",
  "summary": {
    "customer_request_text": "Клиент просит изменить договор",
    "ai_outcome_text": "AI собрал контекст и не выполнял чувствительное действие",
    "recommended_next_step": "Проверить договор и продолжить вручную"
  }
}

Behavior:

• bridge validates the call is still active;
• bridge updates ai_state to handoff_required;
• bridge appends timeline ai.handoff_requested;
• bridge redirects the customer leg into the configured human queue;
• when the normal operator-connected flow happens, the same call row becomes human_owned.

This endpoint is internal-only and trusted for service actors such as svc:ai-voice-runtime.

8.4 interaction_service

Add one internal endpoint:

POST /interactions/{interaction_id}/timeline

Request:

{
  "action": "ai.reply_generated",
  "metadata": {
    "call_id": "1740912000.12",
    "voice_session_id": "avs_...",
    "ai_session_id": "ais_..."
  }
}

Why add this now:

• Voice AI should not write cross-service timeline rows by reaching into another service's DB contract ad hoc;
• the same endpoint can later be reused by Telegram AI without changing its current behavior immediately;
• it makes the AI event contract explicit.

Required Voice AI timeline actions:

• ai.session_started
• ai.reply_generated
• ai.handoff_requested
• ai.handoff_completed
• ai.error

Existing interaction endpoints remain reused as-is:

• PATCH /interactions/{id}/status
• POST /interactions/{id}/escalate
• PATCH /interactions/{id}/assign

8.5 Operator-facing read APIs

Extend asterisk_bridge_service output:

GET /asterisk/live-calls

Add to each row:

• voice_session_id
• ai_session_id
• ai_state
• ai_handoff_reason
• ai_last_model_at

GET /asterisk/recent-calls

Expose the same additive AI fields.

GET /asterisk/live-calls/{call_id}/ai-summary

Return a summary shape intentionally aligned with Telegram:

{
  "call_id": "1740912000.12",
  "session_id": "ais_...",
  "voice_session_id": "avs_...",
  "status_label": "AI передал звонок оператору",
  "status_tone": "handoff",
  "customer_request_text": "Клиент хочет узнать статус обращения и изменить способ оплаты",
  "ai_outcome_text": "AI собрал контекст и объяснил рамки, но не выполнил чувствительное действие",
  "handoff_reason": "Запрос требует человека и проверки вручную",
  "recommended_next_step": "Проверить карточку обращения и продолжить звонок вручную",
  "generated_at": "2026-03-09T12:34:56Z"
}

This should be produced from voice_ai_sessions, ai_turns, and the latest transcript segments.

9. Context Filtering Rules

Voice AI must not send the whole call history into the model each turn.

For every voice turn, ai_orchestrator_service should assemble a filtered context from:

• customer profile:
  • customer_id
  • display name
  • preferred phone
  • tags
• customer memory:
  • recent resolved issues
  • notable preferences
• interaction state:
  • interaction_id
  • status
  • queue
  • last relevant timeline events
• voice session state:
  • language
  • disclosure already played or not
  • previous handoff flag
  • current turn number
• recent transcript window:
  • last 6-10 finalized segments, not the entire transcript
• KB:
  • top 3 matched articles or fewer
• business policy:
  • allowed actions
  • mandatory disclosure
  • sensitive-topic escalation rules

This matches the Telegram AI design principle already present in the repo.

10. Runtime Behavior

10.1 ASR/TTS abstraction

ai_voice_runtime_service should expose provider interfaces and start with one configured provider per environment.

Recommended internal modules:

• providers/asr.py
• providers/tts.py
• session_manager.py
• media_bridge.py
• barge_in.py

V1 supports one active ASR provider and one active TTS provider, behind interfaces. Multi-provider fallback is not required in the first version.

10.2 Barge-in

Barge-in is a V1 requirement because voice UX breaks if the caller cannot interrupt TTS.

Required behavior:

• while TTS is playing, incoming speech activity stops or fades out current playback;
• interrupted assistant output is marked with barge_in_interrupted=true in transcript;
• only finalized caller speech creates a new orchestrator turn;
• if interruption happens repeatedly or ASR confidence is poor, handoff rules may trigger.

10.3 Latency budget

Target budget for one AI turn:

• end-of-utterance to finalized ASR text: <= 600 ms
• orchestrator decision: <= 900 ms
• first TTS audio chunk: <= 500 ms
• total pause before AI speech starts: <= 2.0 s

If the runtime cannot stay within the budget repeatedly, it should prefer human handoff over a degraded long-silence experience.

11. Operator UI Integration

Voice AI V1 should integrate into the current operator shell, not create a second voice console.

11.1 Existing surfaces to extend

• browser call popup overlay;
• Voice debug live/recent calls list;
• unified customer history on /operator.

11.2 Required UI changes

1. Extend live call rows with an AI chip using the same language as Telegram:
   • AI active
   • Ждёт человека
   • AI error
2. When a transferred AI-owned call reaches the operator popup:
   • load GET /asterisk/live-calls/{call_id}/ai-summary;
   • show a compact Сводка AI card above call actions;
   • keep existing Принять в работу, Передать, Завершить controls unchanged.
3. Add AI metadata to customer history:
   • AI session started
   • AI handoff requested
   • AI handoff completed
4. Do not add a separate full transcript workspace in V1.

11.3 UI behavior intentionally deferred

Deferred to V2:

• operator button Вернуть AI for live voice calls;
• inline live transcript for operators during the call;
• supervisor transcript explorer for full recordings plus transcripts.

12. Deployment and Config

12.1 New service in docker-compose.server.yml

Add:

• ai-voice-runtime-service

New shared env:

• AI_VOICE_RUNTIME_SERVICE_URL
• AI_VOICE_ENABLED
• AI_VOICE_QUEUE_CONFIG_JSON
• AI_VOICE_ASR_PROVIDER
• AI_VOICE_TTS_PROVIDER
• AI_VOICE_TTS_CACHE_ENABLED
• AI_VOICE_TTS_CACHE_DIR
• AI_VOICE_MAX_CONTEXT_SEGMENTS
• AI_VOICE_HANDOFF_TIMEOUT_SECONDS

Service auth:

• add trusted subject svc:ai-voice-runtime where internal bridge endpoints require it.

12.2 Asterisk-side change

The accepted human voice path stays intact.

Voice AI adds one new AI media bridge path in dialplan only for AI-selected queues. Preferred V1 implementation is:

• Asterisk dialplan redirects the customer leg into an external media bridge context dedicated to AI.

Exact low-level primitive should be validated in the lab:

• preferred: AudioSocket or equivalent bidirectional audio bridge;
• fallback: narrowly scoped external-media/ARI only for AI queues.

The choice must keep Asterisk as the telephony owner.

13. Rollout Order

Recommended implementation order:

1. schema changes only:
   • voice_ai_sessions
   • voice_transcript_segments
   • additive AI columns on asterisk_call_links
   • additive call_id on ai_sessions
2. control plane only:
   • bridge starts and closes empty runtime sessions behind AI_VOICE_ENABLED=0
3. lab media path:
   • one AI-enabled lab queue
   • disclosure + greeting + ASR/TTS echo flow
4. orchestrator integration:
   • KB lookup
   • filtered context
   • handoff decisioning
5. operator summary:
   • popup card
   • voice debug AI chips
6. staged pilot rollout per queue.

14. Key Risks and Open Questions

1. call_id stability during AI-to-human redirect must be validated in the Asterisk lab. If redirect creates a new call id, bridge recovery must switch to linked_id first and only then reuse interaction_id.
2. The exact Asterisk media primitive must be confirmed before implementation. The design assumes a bidirectional bridge is available without replacing the accepted telephony baseline.
3. Provider latency must be measured in the target environment before enabling AI-first for production queues.
4. Sensitive actions should stay read-only in V1. Voice AI should use KB, customer lookup, and safe interaction updates, but not execute risky external business actions directly.

15. Summary

Voice AI V1 should be implemented as an additive AI layer over the accepted voice baseline:

• asterisk_bridge_service remains telephony truth and handoff executor;
• ai_voice_runtime_service is the new realtime media layer;
• ai_orchestrator_service is reused for deterministic business decisioning;
• interaction_service remains the owner of the canonical AI timeline;
• operator UI gets AI badges and AI handoff summary, not a new telephony product.

This keeps the current live voice contour intact while adding the same AI-first and human-handoff architecture that already works in Telegram.