call-center/docs/architecture/voice-ai-v1.md

# Voice AI V1 Technical Design

Status: `Proposed`

## 1. Goal

Add an AI-first voice operator on top of the accepted Asterisk voice baseline without replacing the current telephony stack.

Voice AI V1 must:

- answer selected inbound calls before a human joins;
- reuse the existing `interaction`, `customer`, `KB`, `queue`, and operator UI model;
- keep Asterisk as the system of record for call lifecycle, transfer, recording, SIP/WebRTC, and queue ownership;
- store business state in platform tables, not only inside model context;
- support deterministic AI-to-human handoff into the existing human queue.

## 2. Non-goals

Voice AI V1 does not:

- replace Asterisk, AMI, dialplan, queueing, SIP/WebRTC, or recording;
- introduce a separate AI telephony platform or second source of truth for calls;
- push full CRM history, full call transcript, or raw recordings into every model turn;
- introduce a multi-agent swarm;
- add attended transfer, conference, or full human-to-AI return in the same live call.

`return-to-ai` after human takeover is intentionally deferred to V2. It requires a second telephony redirect and fresh media re-attachment, which is riskier than AI-first plus human handoff.

## 3. Existing Baseline We Build On

The current repo already has the pieces needed for Voice AI V1:

- `asterisk_bridge_service` owns AMI ingestion, live call tracking, claim/hangup/transfer, and recording import.
- `voice_adapter_service` stores telephony lifecycle events.
- `interaction_service` owns the canonical interaction lifecycle.
- `customer_service` plus `customer_external_identities` already provide cross-channel identity linking.
- `ai_orchestrator_service` already implements the Telegram AI pattern:
  - deterministic orchestration;
  - explicit AI disclosure;
  - `ai_sessions` and `ai_turns`;
  - AI-to-human handoff;
  - source-of-truth separation between channel service and AI service.
- `/operator` already has a hybrid Telegram AI UX with AI badges and AI summary cards.

Voice AI V1 should copy the Telegram AI service boundary pattern, not the Telegram channel model itself.

## 4. Target Architecture

### 4.1 Service responsibilities

| Service | Role in Voice AI V1 |
| --- | --- |
| `asterisk_bridge_service` | Keeps telephony truth, selects AI path for eligible calls, starts/stops AI runtime sessions, performs AI-to-human redirect, exposes operator-facing voice AI summary. |
| `ai_voice_runtime_service` | New realtime service. Owns media session state, ASR/TTS streaming, turn detection, barge-in, runtime latency budget, and handoff requests. |
| `ai_orchestrator_service` | Reused and extended. Owns deterministic voice decisioning, filtered context assembly, KB/tool usage, safe business policies, and AI summary generation. |
| `interaction_service` | Remains the owner of interaction status and timeline; gains an internal timeline append endpoint for additive AI events. |
| `voice_adapter_service` | Stays the owner of telephony event persistence only. It does not become the AI runtime. |
| `ui/operator` | Reuses the current live call and popup flow; adds voice AI badges and handoff summary, but no separate `AI telephony` workspace. |

### 4.2 High-level component view

```mermaid
flowchart LR
    A["Asterisk<br/>queue, redirect, recording, SIP/WebRTC"] --> B["asterisk_bridge_service<br/>call truth + control plane"]
    B --> C["ai_voice_runtime_service<br/>media bridge + ASR/TTS + barge-in"]
    C --> D["ai_orchestrator_service<br/>policy + tools + summaries"]
    D --> E["customer_service / shared DB<br/>customer profile + memory"]
    D --> F["interaction_service<br/>interaction status + timeline"]
    D --> G["kb_service<br/>KB lookup"]
    B --> H["operator UI<br/>live call popup + voice AI summary"]
```

### 4.3 Source-of-truth rule

- Telephony truth stays in `AsteriskCallLinkRow`, AMI events, queue redirect, and recording flow.
- AI truth stays in `voice_ai_sessions`, `ai_sessions`, `ai_turns`, and transcript rows.
- Interaction truth stays in `Interaction` plus `InteractionTimeline`.
- Customer truth stays in `Customer` plus `CustomerExternalIdentity`.

No single prompt becomes the long-term state container.

## 5. Call Flow

### 5.1 AI-first inbound voice flow

```mermaid
sequenceDiagram
    participant A as Asterisk
    participant B as asterisk_bridge_service
    participant R as ai_voice_runtime_service
    participant O as ai_orchestrator_service
    participant I as interaction_service

    A->>B: MVPCCCallStarted
    B->>I: create / reuse interaction
    B->>B: create or update AsteriskCallLinkRow
    B->>R: POST /internal/voice-ai/sessions
    R->>O: POST /ai/voice/sessions/{id}/start
    R->>A: play AI disclosure + greeting
    A->>R: caller audio stream
    R->>O: POST /ai/voice/sessions/{id}/turns
    O-->>R: reply plan or handoff decision
    R->>A: TTS playback
```

Detailed steps:

1. `asterisk_bridge_service` receives the existing `MVPCCCallStarted` event.
2. It keeps the current behavior of creating or resolving `interaction_id` and `AsteriskCallLinkRow`.
3. It evaluates whether this queue or IVR outcome should go to AI-first mode.
4. If not AI-eligible, the current human flow stays unchanged.
5. If AI-eligible:
   - create `voice_ai_session`;
   - create or link generic `ai_session` with `channel="voice"`;
   - mark the call row as AI-owned;
   - start a runtime control session in `ai_voice_runtime_service`;
   - attach the customer leg to the AI media bridge.
6. `ai_voice_runtime_service` plays a short disclosure and greeting.
7. Finalized caller utterances are sent to `ai_orchestrator_service`.
8. The orchestrator loads filtered business context, runs deterministic policy and KB/tool lookup, and returns a structured decision.
9. The runtime either:
   - plays TTS back to the caller; or
   - requests handoff to a human queue.

### 5.2 AI-to-human handoff

```mermaid
sequenceDiagram
    participant R as ai_voice_runtime_service
    participant O as ai_orchestrator_service
    participant B as asterisk_bridge_service
    participant A as Asterisk
    participant U as Operator UI

    R->>O: POST voice turn
    O-->>R: needs_handoff=true + reason + summary
    R->>B: POST /internal/voice-ai/calls/{call_id}/handoff
    B->>B: mark ai_state=handoff_required
    B->>A: redirect caller leg to existing human queue
    A->>U: normal operator incoming call path
    U->>B: GET /asterisk/live-calls/{call_id}/ai-summary
    B-->>U: AI handoff summary
```

Handoff rules:

- handoff happens only through the existing Asterisk queue path;
- the AI runtime never calls an operator directly and never becomes a second queueing owner;
- `interaction_id` stays the same through AI and human phases;
- the operator receives a concise AI summary before or during claim.

### 5.3 Call end

- `MVPCCCallEnded` and `MVPCCRecordingReady` continue to come from Asterisk through the existing bridge.
- The bridge informs the runtime session that telephony ended.
- The runtime closes `voice_ai_session`.
- The orchestrator writes the final AI summary and marks `ai_session` closed.
- The interaction timeline gets final AI events if they were not written yet.

## 6. Queue Selection and AI Eligibility

Voice AI V1 should start with config-driven AI eligibility, not a new admin UI.

Recommended V1 config:

`AI_VOICE_QUEUE_CONFIG_JSON`

Example shape:

```json
{
  "voice_lab": {
    "mode": "ai_first",
    "agent_profile": "voice_support",
    "handoff_queue_code": "voice_lab",
    "language": "ru"
  }
}
```

Why config-first:

- it avoids touching the accepted routing UI and queue schema in the first step;
- it keeps rollout reversible per queue;
- it matches the current repo style where telephony behavior is still largely env-driven.

The longer-term path can move this policy into routing/admin later.

## 7. Data Model

## 7.1 Reused tables

These tables stay authoritative and should be reused:

- `interactions`
- `interaction_timelines`
- `customer_external_identities`
- `asterisk_call_links`
- `voice_events`
- `call_recordings`
- `ai_sessions`
- `ai_turns`

Voice customer linking should reuse `customer_external_identities` with:

- `channel = "voice"`
- `external_subject = normalized caller number`

No new identity table is needed.

## 7.2 New table: `voice_ai_sessions`

This table stores runtime state that does not fit cleanly into generic `ai_sessions`.

Suggested columns:

| Column | Type | Purpose |
| --- | --- | --- |
| `id` | integer pk | Internal row id |
| `session_id` | string unique | Public runtime session id |
| `call_id` | string unique index | Current Asterisk customer-leg call id |
| `linked_id` | string index | Asterisk linked id for recovery if call id changes |
| `interaction_id` | string index | Shared interaction |
| `customer_id` | string index nullable | Linked customer |
| `queue_id` | string index | Original platform queue |
| `ai_session_id` | string index | Generic AI session link |
| `agent_profile` | string index | Voice policy profile |
| `status` | string index | `queued`, `greeting`, `listening`, `thinking`, `speaking`, `handoff_requested`, `human_owned`, `completed`, `error` |
| `language` | string index nullable | Current active language |
| `asr_provider` | string nullable | Selected ASR backend |
| `tts_provider` | string nullable | Selected TTS backend |
| `handoff_reason` | text nullable | Last handoff reason |
| `handoff_target_queue_id` | string nullable | Human target queue |
| `disclosure_played_at` | string nullable | When AI disclosure was first spoken |
| `last_user_utterance_at` | string nullable | Latest finalized caller speech |
| `last_ai_reply_at` | string nullable | Latest completed AI reply |
| `started_at` | string index | Session start |
| `updated_at` | string index | Last state update |
| `ended_at` | string index nullable | Session end |

## 7.3 New table: `voice_transcript_segments`

This table stores finalized voice transcript units separately from generic AI turns.

Suggested columns:

| Column | Type | Purpose |
| --- | --- | --- |
| `id` | integer pk | Internal row id |
| `segment_id` | string unique | Public segment id |
| `session_id` | string index | `voice_ai_sessions.session_id` |
| `call_id` | string index | Voice call correlation |
| `interaction_id` | string index | Interaction correlation |
| `speaker` | string index | `caller`, `assistant`, `system`, `operator` |
| `source_type` | string index | `asr`, `tts`, `handoff_summary`, `system` |
| `sequence_no` | integer index | Ordered segment number |
| `text` | text | Final transcript text |
| `confidence` | number nullable | ASR confidence when applicable |
| `is_final` | boolean | Finalized transcript only in V1, but keep the flag for future partials |
| `barge_in_interrupted` | boolean | Whether the assistant segment was interrupted |
| `payload_json` | text | Provider metadata, timestamps, tool refs |
| `created_at` | string index | Write time |

V1 should store finalized transcript segments only. Partial ASR events can stay in memory inside the runtime.

## 7.4 Additive columns on `asterisk_call_links`

This mirrors the Telegram pattern where the channel source-of-truth row also exposes AI status.

Add:

- `voice_session_id VARCHAR(64) NULL`
- `ai_session_id VARCHAR(64) NULL`
- `ai_state VARCHAR(32) NULL`
- `ai_handoff_reason TEXT NULL`
- `ai_last_model_at VARCHAR(64) NULL`

Suggested `ai_state` values:

- `queued`
- `greeting`
- `listening`
- `thinking`
- `active`
- `handoff_required`
- `human_owned`
- `closed`
- `error`

This lets `/asterisk/live-calls` and `/asterisk/recent-calls` drive UI chips directly without extra joins on every poll.

## 7.5 Additive column on `ai_sessions`

Add:

- `call_id VARCHAR(128) NULL`

Reason:

- Telegram already uses `thread_id`;
- voice needs a direct telephony key for fast lookup and summary generation;
- this keeps `ai_sessions` truly cross-channel instead of Telegram-shaped.

## 7.6 Reuse of `ai_turns`

Reuse `ai_turns` for:

- finalized user turn seen by the orchestrator;
- model reply plan;
- tool invocation results;
- summary turn written during handoff or closure.

Do not reuse `ai_jobs` for voice V1. Telegram jobs are thread-triggered and synchronous voice turn processing does not need that queue model.

## 8. API Design

## 8.1 `asterisk_bridge_service` -> `ai_voice_runtime_service`

New internal control-plane endpoints:

### `POST /internal/voice-ai/sessions`

Starts a runtime session for an already accepted telephony call.

Request:

```json
{
  "call_id": "1740912000.12",
  "linked_id": "1740912000.12",
  "interaction_id": "int_...",
  "queue_id": "que_...",
  "caller_number": "+7701...",
  "caller_name": "Lab Caller",
  "agent_profile": "voice_support",
  "language_hint": "ru",
  "handoff_queue_id": "que_...",
  "metadata": {
    "queue_code": "voice_lab",
    "direction": "inbound"
  }
}
```

Response:

```json
{
  "voice_session_id": "avs_...",
  "ai_session_id": "ais_...",
  "status": "queued"
}
```

### `POST /internal/voice-ai/sessions/{session_id}/telephony-events`

Bridge notifies runtime about:

- `call.connected`
- `call.ended`
- `recording.ready`
- `operator.connected`

This keeps telephony truth in the bridge while runtime stays current.

## 8.2 `ai_voice_runtime_service` -> `ai_orchestrator_service`

### `POST /ai/voice/sessions/{session_id}/start`

Creates or reopens the generic AI session and returns greeting policy:

```json
{
  "voice_session_id": "avs_...",
  "call_id": "1740912000.12",
  "interaction_id": "int_...",
  "customer_id": "cus_...",
  "language_hint": "ru",
  "agent_profile": "voice_support"
}
```

Response:

```json
{
  "session_id": "ais_...",
  "language": "ru",
  "greeting_text": "Здравствуйте. Я AI-оператор компании...",
  "disclosure_required": true
}
```

### `POST /ai/voice/sessions/{session_id}/turns`

Main deterministic decision endpoint.

Request:

```json
{
  "voice_session_id": "avs_...",
  "call_id": "1740912000.12",
  "interaction_id": "int_...",
  "transcript_text": "Хочу узнать статус заявки",
  "language": "ru",
  "sequence_no": 3,
  "barge_in": false,
  "metadata": {
    "turn_duration_ms": 4200
  }
}
```

Response:

```json
{
  "language": "ru",
  "intent": "status_check",
  "reply_text": "Я AI-оператор компании. Проверяю данные по обращению...",
  "confidence": 0.83,
  "needs_handoff": false,
  "handoff_reason": null,
  "case_action": "keep_open",
  "kb_refs": ["art_..."],
  "summary_text": "Клиент уточняет статус заявки.",
  "model": "gpt-4o-mini",
  "latency_ms": 780
}
```

`case_action` should stay aligned with the existing Telegram pattern:

- `none`
- `keep_open`
- `close`
- `escalate`

### `POST /ai/voice/sessions/{session_id}/close`

Finalizes AI state and writes the terminal summary.

## 8.3 `ai_voice_runtime_service` -> `asterisk_bridge_service`

### `POST /internal/voice-ai/calls/{call_id}/handoff`

Requests redirect of the current customer leg into the existing human queue.

Request:

```json
{
  "voice_session_id": "avs_...",
  "ai_session_id": "ais_...",
  "interaction_id": "int_...",
  "target_queue_id": "que_...",
  "reason": "Нужен человек для чувствительного запроса.",
  "summary": {
    "customer_request_text": "Клиент просит изменить договор",
    "ai_outcome_text": "AI собрал контекст и не выполнял чувствительное действие",
    "recommended_next_step": "Проверить договор и продолжить вручную"
  }
}
```

Behavior:

- bridge validates the call is still active;
- bridge updates `ai_state` to `handoff_required`;
- bridge appends timeline `ai.handoff_requested`;
- bridge redirects the customer leg into the configured human queue;
- when the normal operator-connected flow happens, the same call row becomes `human_owned`.

This endpoint is internal-only and trusted for service actors such as `svc:ai-voice-runtime`.

## 8.4 `interaction_service`

Add one internal endpoint:

### `POST /interactions/{interaction_id}/timeline`

Request:

```json
{
  "action": "ai.reply_generated",
  "metadata": {
    "call_id": "1740912000.12",
    "voice_session_id": "avs_...",
    "ai_session_id": "ais_..."
  }
}
```

Why add this now:

- Voice AI should not write cross-service timeline rows by reaching into another service's DB contract ad hoc;
- the same endpoint can later be reused by Telegram AI without changing its current behavior immediately;
- it makes the AI event contract explicit.

Required Voice AI timeline actions:

- `ai.session_started`
- `ai.reply_generated`
- `ai.handoff_requested`
- `ai.handoff_completed`
- `ai.error`

Existing interaction endpoints remain reused as-is:

- `PATCH /interactions/{id}/status`
- `POST /interactions/{id}/escalate`
- `PATCH /interactions/{id}/assign`

## 8.5 Operator-facing read APIs

Extend `asterisk_bridge_service` output:

### `GET /asterisk/live-calls`

Add to each row:

- `voice_session_id`
- `ai_session_id`
- `ai_state`
- `ai_handoff_reason`
- `ai_last_model_at`

### `GET /asterisk/recent-calls`

Expose the same additive AI fields.

### `GET /asterisk/live-calls/{call_id}/ai-summary`

Return a summary shape intentionally aligned with Telegram:

```json
{
  "call_id": "1740912000.12",
  "session_id": "ais_...",
  "voice_session_id": "avs_...",
  "status_label": "AI передал звонок оператору",
  "status_tone": "handoff",
  "customer_request_text": "Клиент хочет узнать статус обращения и изменить способ оплаты",
  "ai_outcome_text": "AI собрал контекст и объяснил рамки, но не выполнил чувствительное действие",
  "handoff_reason": "Запрос требует человека и проверки вручную",
  "recommended_next_step": "Проверить карточку обращения и продолжить звонок вручную",
  "generated_at": "2026-03-09T12:34:56Z"
}
```

This should be produced from `voice_ai_sessions`, `ai_turns`, and the latest transcript segments.

## 9. Context Filtering Rules

Voice AI must not send the whole call history into the model each turn.

For every voice turn, `ai_orchestrator_service` should assemble a filtered context from:

- customer profile:
  - `customer_id`
  - display name
  - preferred phone
  - tags
- customer memory:
  - recent resolved issues
  - notable preferences
- interaction state:
  - `interaction_id`
  - status
  - queue
  - last relevant timeline events
- voice session state:
  - language
  - disclosure already played or not
  - previous handoff flag
  - current turn number
- recent transcript window:
  - last `6-10` finalized segments, not the entire transcript
- KB:
  - top `3` matched articles or fewer
- business policy:
  - allowed actions
  - mandatory disclosure
  - sensitive-topic escalation rules

This matches the Telegram AI design principle already present in the repo.

## 10. Runtime Behavior

## 10.1 ASR/TTS abstraction

`ai_voice_runtime_service` should expose provider interfaces and start with one configured provider per environment.

Recommended internal modules:

- `providers/asr.py`
- `providers/tts.py`
- `session_manager.py`
- `media_bridge.py`
- `barge_in.py`

V1 supports one active ASR provider and one active TTS provider, behind interfaces. Multi-provider fallback is not required in the first version.

## 10.2 Barge-in

Barge-in is a V1 requirement because voice UX breaks if the caller cannot interrupt TTS.

Required behavior:

- while TTS is playing, incoming speech activity stops or fades out current playback;
- interrupted assistant output is marked with `barge_in_interrupted=true` in transcript;
- only finalized caller speech creates a new orchestrator turn;
- if interruption happens repeatedly or ASR confidence is poor, handoff rules may trigger.

## 10.3 Latency budget

Target budget for one AI turn:

- end-of-utterance to finalized ASR text: `<= 600 ms`
- orchestrator decision: `<= 900 ms`
- first TTS audio chunk: `<= 500 ms`
- total pause before AI speech starts: `<= 2.0 s`

If the runtime cannot stay within the budget repeatedly, it should prefer human handoff over a degraded long-silence experience.

## 11. Operator UI Integration

Voice AI V1 should integrate into the current operator shell, not create a second voice console.

### 11.1 Existing surfaces to extend

- browser call popup overlay;
- `Voice debug` live/recent calls list;
- unified customer history on `/operator`.

### 11.2 Required UI changes

1. Extend live call rows with an AI chip using the same language as Telegram:
   - `AI active`
   - `Ждёт человека`
   - `AI error`
2. When a transferred AI-owned call reaches the operator popup:
   - load `GET /asterisk/live-calls/{call_id}/ai-summary`;
   - show a compact `Сводка AI` card above call actions;
   - keep existing `Принять в работу`, `Передать`, `Завершить` controls unchanged.
3. Add AI metadata to customer history:
   - AI session started
   - AI handoff requested
   - AI handoff completed
4. Do not add a separate full transcript workspace in V1.

### 11.3 UI behavior intentionally deferred

Deferred to V2:

- operator button `Вернуть AI` for live voice calls;
- inline live transcript for operators during the call;
- supervisor transcript explorer for full recordings plus transcripts.

## 12. Deployment and Config

## 12.1 New service in `docker-compose.server.yml`

Add:

- `ai-voice-runtime-service`

New shared env:

- `AI_VOICE_RUNTIME_SERVICE_URL`
- `AI_VOICE_ENABLED`
- `AI_VOICE_QUEUE_CONFIG_JSON`
- `AI_VOICE_ASR_PROVIDER`
- `AI_VOICE_TTS_PROVIDER`
- `AI_VOICE_TTS_CACHE_ENABLED`
- `AI_VOICE_TTS_CACHE_DIR`
- `AI_VOICE_MAX_CONTEXT_SEGMENTS`
- `AI_VOICE_HANDOFF_TIMEOUT_SECONDS`

Service auth:

- add trusted subject `svc:ai-voice-runtime` where internal bridge endpoints require it.

## 12.2 Asterisk-side change

The accepted human voice path stays intact.

Voice AI adds one new AI media bridge path in dialplan only for AI-selected queues. Preferred V1 implementation is:

- Asterisk dialplan redirects the customer leg into an external media bridge context dedicated to AI.

Exact low-level primitive should be validated in the lab:

- preferred: `AudioSocket` or equivalent bidirectional audio bridge;
- fallback: narrowly scoped external-media/ARI only for AI queues.

The choice must keep Asterisk as the telephony owner.

## 13. Rollout Order

Recommended implementation order:

1. schema changes only:
   - `voice_ai_sessions`
   - `voice_transcript_segments`
   - additive AI columns on `asterisk_call_links`
   - additive `call_id` on `ai_sessions`
2. control plane only:
   - bridge starts and closes empty runtime sessions behind `AI_VOICE_ENABLED=0`
3. lab media path:
   - one AI-enabled lab queue
   - disclosure + greeting + ASR/TTS echo flow
4. orchestrator integration:
   - KB lookup
   - filtered context
   - handoff decisioning
5. operator summary:
   - popup card
   - voice debug AI chips
6. staged pilot rollout per queue.

## 14. Key Risks and Open Questions

1. `call_id` stability during AI-to-human redirect must be validated in the Asterisk lab.
   If redirect creates a new call id, bridge recovery must switch to `linked_id` first and only then reuse `interaction_id`.
2. The exact Asterisk media primitive must be confirmed before implementation.
   The design assumes a bidirectional bridge is available without replacing the accepted telephony baseline.
3. Provider latency must be measured in the target environment before enabling AI-first for production queues.
4. Sensitive actions should stay read-only in V1.
   Voice AI should use KB, customer lookup, and safe interaction updates, but not execute risky external business actions directly.

## 15. Summary

Voice AI V1 should be implemented as an additive AI layer over the accepted voice baseline:

- `asterisk_bridge_service` remains telephony truth and handoff executor;
- `ai_voice_runtime_service` is the new realtime media layer;
- `ai_orchestrator_service` is reused for deterministic business decisioning;
- `interaction_service` remains the owner of the canonical AI timeline;
- operator UI gets AI badges and AI handoff summary, not a new telephony product.

This keeps the current live voice contour intact while adding the same AI-first and human-handoff architecture that already works in Telegram.