|
|
|
@@ -0,0 +1,744 @@
|
|
|
|
|
# Voice AI V1 Technical Design
|
|
|
|
|
|
|
|
|
|
Status: `Proposed`
|
|
|
|
|
|
|
|
|
|
## 1. Goal
|
|
|
|
|
|
|
|
|
|
Add an AI-first voice operator on top of the accepted Asterisk voice baseline without replacing the current telephony stack.
|
|
|
|
|
|
|
|
|
|
Voice AI V1 must:
|
|
|
|
|
|
|
|
|
|
- answer selected inbound calls before a human joins;
|
|
|
|
|
- reuse the existing `interaction`, `customer`, `KB`, `queue`, and operator UI model;
|
|
|
|
|
- keep Asterisk as the system of record for call lifecycle, transfer, recording, SIP/WebRTC, and queue ownership;
|
|
|
|
|
- store business state in platform tables, not only inside model context;
|
|
|
|
|
- support deterministic AI-to-human handoff into the existing human queue.
|
|
|
|
|
|
|
|
|
|
## 2. Non-goals
|
|
|
|
|
|
|
|
|
|
Voice AI V1 does not:
|
|
|
|
|
|
|
|
|
|
- replace Asterisk, AMI, dialplan, queueing, SIP/WebRTC, or recording;
|
|
|
|
|
- introduce a separate AI telephony platform or second source of truth for calls;
|
|
|
|
|
- push full CRM history, full call transcript, or raw recordings into every model turn;
|
|
|
|
|
- introduce a multi-agent swarm;
|
|
|
|
|
- add attended transfer, conference, or full human-to-AI return in the same live call.
|
|
|
|
|
|
|
|
|
|
`return-to-ai` after human takeover is intentionally deferred to V2. It requires a second telephony redirect and fresh media re-attachment, which is riskier than AI-first plus human handoff.
|
|
|
|
|
|
|
|
|
|
## 3. Existing Baseline We Build On
|
|
|
|
|
|
|
|
|
|
The current repo already has the pieces needed for Voice AI V1:
|
|
|
|
|
|
|
|
|
|
- `asterisk_bridge_service` owns AMI ingestion, live call tracking, claim/hangup/transfer, and recording import.
|
|
|
|
|
- `voice_adapter_service` stores telephony lifecycle events.
|
|
|
|
|
- `interaction_service` owns the canonical interaction lifecycle.
|
|
|
|
|
- `customer_service` plus `customer_external_identities` already provide cross-channel identity linking.
|
|
|
|
|
- `ai_orchestrator_service` already implements the Telegram AI pattern:
|
|
|
|
|
- deterministic orchestration;
|
|
|
|
|
- explicit AI disclosure;
|
|
|
|
|
- `ai_sessions` and `ai_turns`;
|
|
|
|
|
- AI-to-human handoff;
|
|
|
|
|
- source-of-truth separation between channel service and AI service.
|
|
|
|
|
- `/operator` already has a hybrid Telegram AI UX with AI badges and AI summary cards.
|
|
|
|
|
|
|
|
|
|
Voice AI V1 should copy the Telegram AI service boundary pattern, not the Telegram channel model itself.
|
|
|
|
|
|
|
|
|
|
## 4. Target Architecture
|
|
|
|
|
|
|
|
|
|
### 4.1 Service responsibilities
|
|
|
|
|
|
|
|
|
|
| Service | Role in Voice AI V1 |
|
|
|
|
|
| --- | --- |
|
|
|
|
|
| `asterisk_bridge_service` | Keeps telephony truth, selects AI path for eligible calls, starts/stops AI runtime sessions, performs AI-to-human redirect, exposes operator-facing voice AI summary. |
|
|
|
|
|
| `ai_voice_runtime_service` | New realtime service. Owns media session state, ASR/TTS streaming, turn detection, barge-in, runtime latency budget, and handoff requests. |
|
|
|
|
|
| `ai_orchestrator_service` | Reused and extended. Owns deterministic voice decisioning, filtered context assembly, KB/tool usage, safe business policies, and AI summary generation. |
|
|
|
|
|
| `interaction_service` | Remains the owner of interaction status and timeline; gains an internal timeline append endpoint for additive AI events. |
|
|
|
|
|
| `voice_adapter_service` | Stays the owner of telephony event persistence only. It does not become the AI runtime. |
|
|
|
|
|
| `ui/operator` | Reuses the current live call and popup flow; adds voice AI badges and handoff summary, but no separate `AI telephony` workspace. |
|
|
|
|
|
|
|
|
|
|
### 4.2 High-level component view
|
|
|
|
|
|
|
|
|
|
```mermaid
|
|
|
|
|
flowchart LR
|
|
|
|
|
A["Asterisk<br/>queue, redirect, recording, SIP/WebRTC"] --> B["asterisk_bridge_service<br/>call truth + control plane"]
|
|
|
|
|
B --> C["ai_voice_runtime_service<br/>media bridge + ASR/TTS + barge-in"]
|
|
|
|
|
C --> D["ai_orchestrator_service<br/>policy + tools + summaries"]
|
|
|
|
|
D --> E["customer_service / shared DB<br/>customer profile + memory"]
|
|
|
|
|
D --> F["interaction_service<br/>interaction status + timeline"]
|
|
|
|
|
D --> G["kb_service<br/>KB lookup"]
|
|
|
|
|
B --> H["operator UI<br/>live call popup + voice AI summary"]
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### 4.3 Source-of-truth rule
|
|
|
|
|
|
|
|
|
|
- Telephony truth stays in `AsteriskCallLinkRow`, AMI events, queue redirect, and recording flow.
|
|
|
|
|
- AI truth stays in `voice_ai_sessions`, `ai_sessions`, `ai_turns`, and transcript rows.
|
|
|
|
|
- Interaction truth stays in `Interaction` plus `InteractionTimeline`.
|
|
|
|
|
- Customer truth stays in `Customer` plus `CustomerExternalIdentity`.
|
|
|
|
|
|
|
|
|
|
No single prompt becomes the long-term state container.
|
|
|
|
|
|
|
|
|
|
## 5. Call Flow
|
|
|
|
|
|
|
|
|
|
### 5.1 AI-first inbound voice flow
|
|
|
|
|
|
|
|
|
|
```mermaid
|
|
|
|
|
sequenceDiagram
|
|
|
|
|
participant A as Asterisk
|
|
|
|
|
participant B as asterisk_bridge_service
|
|
|
|
|
participant R as ai_voice_runtime_service
|
|
|
|
|
participant O as ai_orchestrator_service
|
|
|
|
|
participant I as interaction_service
|
|
|
|
|
|
|
|
|
|
A->>B: MVPCCCallStarted
|
|
|
|
|
B->>I: create / reuse interaction
|
|
|
|
|
B->>B: create or update AsteriskCallLinkRow
|
|
|
|
|
B->>R: POST /internal/voice-ai/sessions
|
|
|
|
|
R->>O: POST /ai/voice/sessions/{id}/start
|
|
|
|
|
R->>A: play AI disclosure + greeting
|
|
|
|
|
A->>R: caller audio stream
|
|
|
|
|
R->>O: POST /ai/voice/sessions/{id}/turns
|
|
|
|
|
O-->>R: reply plan or handoff decision
|
|
|
|
|
R->>A: TTS playback
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Detailed steps:
|
|
|
|
|
|
|
|
|
|
1. `asterisk_bridge_service` receives the existing `MVPCCCallStarted` event.
|
|
|
|
|
2. It keeps the current behavior of creating or resolving `interaction_id` and `AsteriskCallLinkRow`.
|
|
|
|
|
3. It evaluates whether this queue or IVR outcome should go to AI-first mode.
|
|
|
|
|
4. If not AI-eligible, the current human flow stays unchanged.
|
|
|
|
|
5. If AI-eligible:
|
|
|
|
|
- create `voice_ai_session`;
|
|
|
|
|
- create or link generic `ai_session` with `channel="voice"`;
|
|
|
|
|
- mark the call row as AI-owned;
|
|
|
|
|
- start a runtime control session in `ai_voice_runtime_service`;
|
|
|
|
|
- attach the customer leg to the AI media bridge.
|
|
|
|
|
6. `ai_voice_runtime_service` plays a short disclosure and greeting.
|
|
|
|
|
7. Finalized caller utterances are sent to `ai_orchestrator_service`.
|
|
|
|
|
8. The orchestrator loads filtered business context, runs deterministic policy and KB/tool lookup, and returns a structured decision.
|
|
|
|
|
9. The runtime either:
|
|
|
|
|
- plays TTS back to the caller; or
|
|
|
|
|
- requests handoff to a human queue.
|
|
|
|
|
|
|
|
|
|
### 5.2 AI-to-human handoff
|
|
|
|
|
|
|
|
|
|
```mermaid
|
|
|
|
|
sequenceDiagram
|
|
|
|
|
participant R as ai_voice_runtime_service
|
|
|
|
|
participant O as ai_orchestrator_service
|
|
|
|
|
participant B as asterisk_bridge_service
|
|
|
|
|
participant A as Asterisk
|
|
|
|
|
participant U as Operator UI
|
|
|
|
|
|
|
|
|
|
R->>O: POST voice turn
|
|
|
|
|
O-->>R: needs_handoff=true + reason + summary
|
|
|
|
|
R->>B: POST /internal/voice-ai/calls/{call_id}/handoff
|
|
|
|
|
B->>B: mark ai_state=handoff_required
|
|
|
|
|
B->>A: redirect caller leg to existing human queue
|
|
|
|
|
A->>U: normal operator incoming call path
|
|
|
|
|
U->>B: GET /asterisk/live-calls/{call_id}/ai-summary
|
|
|
|
|
B-->>U: AI handoff summary
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Handoff rules:
|
|
|
|
|
|
|
|
|
|
- handoff happens only through the existing Asterisk queue path;
|
|
|
|
|
- the AI runtime never calls an operator directly and never becomes a second queueing owner;
|
|
|
|
|
- `interaction_id` stays the same through AI and human phases;
|
|
|
|
|
- the operator receives a concise AI summary before or during claim.
|
|
|
|
|
|
|
|
|
|
### 5.3 Call end
|
|
|
|
|
|
|
|
|
|
- `MVPCCCallEnded` and `MVPCCRecordingReady` continue to come from Asterisk through the existing bridge.
|
|
|
|
|
- The bridge informs the runtime session that telephony ended.
|
|
|
|
|
- The runtime closes `voice_ai_session`.
|
|
|
|
|
- The orchestrator writes the final AI summary and marks `ai_session` closed.
|
|
|
|
|
- The interaction timeline gets final AI events if they were not written yet.
|
|
|
|
|
|
|
|
|
|
## 6. Queue Selection and AI Eligibility
|
|
|
|
|
|
|
|
|
|
Voice AI V1 should start with config-driven AI eligibility, not a new admin UI.
|
|
|
|
|
|
|
|
|
|
Recommended V1 config:
|
|
|
|
|
|
|
|
|
|
`AI_VOICE_QUEUE_CONFIG_JSON`
|
|
|
|
|
|
|
|
|
|
Example shape:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"voice_lab": {
|
|
|
|
|
"mode": "ai_first",
|
|
|
|
|
"agent_profile": "voice_support",
|
|
|
|
|
"handoff_queue_code": "voice_lab",
|
|
|
|
|
"language": "ru"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Why config-first:
|
|
|
|
|
|
|
|
|
|
- it avoids touching the accepted routing UI and queue schema in the first step;
|
|
|
|
|
- it keeps rollout reversible per queue;
|
|
|
|
|
- it matches the current repo style where telephony behavior is still largely env-driven.
|
|
|
|
|
|
|
|
|
|
The longer-term path can move this policy into routing/admin later.
|
|
|
|
|
|
|
|
|
|
## 7. Data Model
|
|
|
|
|
|
|
|
|
|
## 7.1 Reused tables
|
|
|
|
|
|
|
|
|
|
These tables stay authoritative and should be reused:
|
|
|
|
|
|
|
|
|
|
- `interactions`
|
|
|
|
|
- `interaction_timelines`
|
|
|
|
|
- `customer_external_identities`
|
|
|
|
|
- `asterisk_call_links`
|
|
|
|
|
- `voice_events`
|
|
|
|
|
- `call_recordings`
|
|
|
|
|
- `ai_sessions`
|
|
|
|
|
- `ai_turns`
|
|
|
|
|
|
|
|
|
|
Voice customer linking should reuse `customer_external_identities` with:
|
|
|
|
|
|
|
|
|
|
- `channel = "voice"`
|
|
|
|
|
- `external_subject = normalized caller number`
|
|
|
|
|
|
|
|
|
|
No new identity table is needed.
|
|
|
|
|
|
|
|
|
|
## 7.2 New table: `voice_ai_sessions`
|
|
|
|
|
|
|
|
|
|
This table stores runtime state that does not fit cleanly into generic `ai_sessions`.
|
|
|
|
|
|
|
|
|
|
Suggested columns:
|
|
|
|
|
|
|
|
|
|
| Column | Type | Purpose |
|
|
|
|
|
| --- | --- | --- |
|
|
|
|
|
| `id` | integer pk | Internal row id |
|
|
|
|
|
| `session_id` | string unique | Public runtime session id |
|
|
|
|
|
| `call_id` | string unique index | Current Asterisk customer-leg call id |
|
|
|
|
|
| `linked_id` | string index | Asterisk linked id for recovery if call id changes |
|
|
|
|
|
| `interaction_id` | string index | Shared interaction |
|
|
|
|
|
| `customer_id` | string index nullable | Linked customer |
|
|
|
|
|
| `queue_id` | string index | Original platform queue |
|
|
|
|
|
| `ai_session_id` | string index | Generic AI session link |
|
|
|
|
|
| `agent_profile` | string index | Voice policy profile |
|
|
|
|
|
| `status` | string index | `queued`, `greeting`, `listening`, `thinking`, `speaking`, `handoff_requested`, `human_owned`, `completed`, `error` |
|
|
|
|
|
| `language` | string index nullable | Current active language |
|
|
|
|
|
| `asr_provider` | string nullable | Selected ASR backend |
|
|
|
|
|
| `tts_provider` | string nullable | Selected TTS backend |
|
|
|
|
|
| `handoff_reason` | text nullable | Last handoff reason |
|
|
|
|
|
| `handoff_target_queue_id` | string nullable | Human target queue |
|
|
|
|
|
| `disclosure_played_at` | string nullable | When AI disclosure was first spoken |
|
|
|
|
|
| `last_user_utterance_at` | string nullable | Latest finalized caller speech |
|
|
|
|
|
| `last_ai_reply_at` | string nullable | Latest completed AI reply |
|
|
|
|
|
| `started_at` | string index | Session start |
|
|
|
|
|
| `updated_at` | string index | Last state update |
|
|
|
|
|
| `ended_at` | string index nullable | Session end |
|
|
|
|
|
|
|
|
|
|
## 7.3 New table: `voice_transcript_segments`
|
|
|
|
|
|
|
|
|
|
This table stores finalized voice transcript units separately from generic AI turns.
|
|
|
|
|
|
|
|
|
|
Suggested columns:
|
|
|
|
|
|
|
|
|
|
| Column | Type | Purpose |
|
|
|
|
|
| --- | --- | --- |
|
|
|
|
|
| `id` | integer pk | Internal row id |
|
|
|
|
|
| `segment_id` | string unique | Public segment id |
|
|
|
|
|
| `session_id` | string index | `voice_ai_sessions.session_id` |
|
|
|
|
|
| `call_id` | string index | Voice call correlation |
|
|
|
|
|
| `interaction_id` | string index | Interaction correlation |
|
|
|
|
|
| `speaker` | string index | `caller`, `assistant`, `system`, `operator` |
|
|
|
|
|
| `source_type` | string index | `asr`, `tts`, `handoff_summary`, `system` |
|
|
|
|
|
| `sequence_no` | integer index | Ordered segment number |
|
|
|
|
|
| `text` | text | Final transcript text |
|
|
|
|
|
| `confidence` | number nullable | ASR confidence when applicable |
|
|
|
|
|
| `is_final` | boolean | Finalized transcript only in V1, but keep the flag for future partials |
|
|
|
|
|
| `barge_in_interrupted` | boolean | Whether the assistant segment was interrupted |
|
|
|
|
|
| `payload_json` | text | Provider metadata, timestamps, tool refs |
|
|
|
|
|
| `created_at` | string index | Write time |
|
|
|
|
|
|
|
|
|
|
V1 should store finalized transcript segments only. Partial ASR events can stay in memory inside the runtime.
|
|
|
|
|
|
|
|
|
|
## 7.4 Additive columns on `asterisk_call_links`
|
|
|
|
|
|
|
|
|
|
This mirrors the Telegram pattern where the channel source-of-truth row also exposes AI status.
|
|
|
|
|
|
|
|
|
|
Add:
|
|
|
|
|
|
|
|
|
|
- `voice_session_id VARCHAR(64) NULL`
|
|
|
|
|
- `ai_session_id VARCHAR(64) NULL`
|
|
|
|
|
- `ai_state VARCHAR(32) NULL`
|
|
|
|
|
- `ai_handoff_reason TEXT NULL`
|
|
|
|
|
- `ai_last_model_at VARCHAR(64) NULL`
|
|
|
|
|
|
|
|
|
|
Suggested `ai_state` values:
|
|
|
|
|
|
|
|
|
|
- `queued`
|
|
|
|
|
- `greeting`
|
|
|
|
|
- `listening`
|
|
|
|
|
- `thinking`
|
|
|
|
|
- `active`
|
|
|
|
|
- `handoff_required`
|
|
|
|
|
- `human_owned`
|
|
|
|
|
- `closed`
|
|
|
|
|
- `error`
|
|
|
|
|
|
|
|
|
|
This lets `/asterisk/live-calls` and `/asterisk/recent-calls` drive UI chips directly without extra joins on every poll.
|
|
|
|
|
|
|
|
|
|
## 7.5 Additive column on `ai_sessions`
|
|
|
|
|
|
|
|
|
|
Add:
|
|
|
|
|
|
|
|
|
|
- `call_id VARCHAR(128) NULL`
|
|
|
|
|
|
|
|
|
|
Reason:
|
|
|
|
|
|
|
|
|
|
- Telegram already uses `thread_id`;
|
|
|
|
|
- voice needs a direct telephony key for fast lookup and summary generation;
|
|
|
|
|
- this keeps `ai_sessions` truly cross-channel instead of Telegram-shaped.
|
|
|
|
|
|
|
|
|
|
## 7.6 Reuse of `ai_turns`
|
|
|
|
|
|
|
|
|
|
Reuse `ai_turns` for:
|
|
|
|
|
|
|
|
|
|
- finalized user turn seen by the orchestrator;
|
|
|
|
|
- model reply plan;
|
|
|
|
|
- tool invocation results;
|
|
|
|
|
- summary turn written during handoff or closure.
|
|
|
|
|
|
|
|
|
|
Do not reuse `ai_jobs` for voice V1. Telegram jobs are thread-triggered and synchronous voice turn processing does not need that queue model.
|
|
|
|
|
|
|
|
|
|
## 8. API Design
|
|
|
|
|
|
|
|
|
|
## 8.1 `asterisk_bridge_service` -> `ai_voice_runtime_service`
|
|
|
|
|
|
|
|
|
|
New internal control-plane endpoints:
|
|
|
|
|
|
|
|
|
|
### `POST /internal/voice-ai/sessions`
|
|
|
|
|
|
|
|
|
|
Starts a runtime session for an already accepted telephony call.
|
|
|
|
|
|
|
|
|
|
Request:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"call_id": "1740912000.12",
|
|
|
|
|
"linked_id": "1740912000.12",
|
|
|
|
|
"interaction_id": "int_...",
|
|
|
|
|
"queue_id": "que_...",
|
|
|
|
|
"caller_number": "+7701...",
|
|
|
|
|
"caller_name": "Lab Caller",
|
|
|
|
|
"agent_profile": "voice_support",
|
|
|
|
|
"language_hint": "ru",
|
|
|
|
|
"handoff_queue_id": "que_...",
|
|
|
|
|
"metadata": {
|
|
|
|
|
"queue_code": "voice_lab",
|
|
|
|
|
"direction": "inbound"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Response:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"voice_session_id": "avs_...",
|
|
|
|
|
"ai_session_id": "ais_...",
|
|
|
|
|
"status": "queued"
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### `POST /internal/voice-ai/sessions/{session_id}/telephony-events`
|
|
|
|
|
|
|
|
|
|
Bridge notifies runtime about:
|
|
|
|
|
|
|
|
|
|
- `call.connected`
|
|
|
|
|
- `call.ended`
|
|
|
|
|
- `recording.ready`
|
|
|
|
|
- `operator.connected`
|
|
|
|
|
|
|
|
|
|
This keeps telephony truth in the bridge while runtime stays current.
|
|
|
|
|
|
|
|
|
|
## 8.2 `ai_voice_runtime_service` -> `ai_orchestrator_service`
|
|
|
|
|
|
|
|
|
|
### `POST /ai/voice/sessions/{session_id}/start`
|
|
|
|
|
|
|
|
|
|
Creates or reopens the generic AI session and returns greeting policy:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"voice_session_id": "avs_...",
|
|
|
|
|
"call_id": "1740912000.12",
|
|
|
|
|
"interaction_id": "int_...",
|
|
|
|
|
"customer_id": "cus_...",
|
|
|
|
|
"language_hint": "ru",
|
|
|
|
|
"agent_profile": "voice_support"
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Response:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"session_id": "ais_...",
|
|
|
|
|
"language": "ru",
|
|
|
|
|
"greeting_text": "Здравствуйте. Я AI-оператор компании...",
|
|
|
|
|
"disclosure_required": true
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### `POST /ai/voice/sessions/{session_id}/turns`
|
|
|
|
|
|
|
|
|
|
Main deterministic decision endpoint.
|
|
|
|
|
|
|
|
|
|
Request:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"voice_session_id": "avs_...",
|
|
|
|
|
"call_id": "1740912000.12",
|
|
|
|
|
"interaction_id": "int_...",
|
|
|
|
|
"transcript_text": "Хочу узнать статус заявки",
|
|
|
|
|
"language": "ru",
|
|
|
|
|
"sequence_no": 3,
|
|
|
|
|
"barge_in": false,
|
|
|
|
|
"metadata": {
|
|
|
|
|
"turn_duration_ms": 4200
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Response:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"language": "ru",
|
|
|
|
|
"intent": "status_check",
|
|
|
|
|
"reply_text": "Я AI-оператор компании. Проверяю данные по обращению...",
|
|
|
|
|
"confidence": 0.83,
|
|
|
|
|
"needs_handoff": false,
|
|
|
|
|
"handoff_reason": null,
|
|
|
|
|
"case_action": "keep_open",
|
|
|
|
|
"kb_refs": ["art_..."],
|
|
|
|
|
"summary_text": "Клиент уточняет статус заявки.",
|
|
|
|
|
"model": "gpt-4o-mini",
|
|
|
|
|
"latency_ms": 780
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
`case_action` should stay aligned with the existing Telegram pattern:
|
|
|
|
|
|
|
|
|
|
- `none`
|
|
|
|
|
- `keep_open`
|
|
|
|
|
- `close`
|
|
|
|
|
- `escalate`
|
|
|
|
|
|
|
|
|
|
### `POST /ai/voice/sessions/{session_id}/close`
|
|
|
|
|
|
|
|
|
|
Finalizes AI state and writes the terminal summary.
|
|
|
|
|
|
|
|
|
|
## 8.3 `ai_voice_runtime_service` -> `asterisk_bridge_service`
|
|
|
|
|
|
|
|
|
|
### `POST /internal/voice-ai/calls/{call_id}/handoff`
|
|
|
|
|
|
|
|
|
|
Requests redirect of the current customer leg into the existing human queue.
|
|
|
|
|
|
|
|
|
|
Request:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"voice_session_id": "avs_...",
|
|
|
|
|
"ai_session_id": "ais_...",
|
|
|
|
|
"interaction_id": "int_...",
|
|
|
|
|
"target_queue_id": "que_...",
|
|
|
|
|
"reason": "Нужен человек для чувствительного запроса.",
|
|
|
|
|
"summary": {
|
|
|
|
|
"customer_request_text": "Клиент просит изменить договор",
|
|
|
|
|
"ai_outcome_text": "AI собрал контекст и не выполнял чувствительное действие",
|
|
|
|
|
"recommended_next_step": "Проверить договор и продолжить вручную"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Behavior:
|
|
|
|
|
|
|
|
|
|
- bridge validates the call is still active;
|
|
|
|
|
- bridge updates `ai_state` to `handoff_required`;
|
|
|
|
|
- bridge appends timeline `ai.handoff_requested`;
|
|
|
|
|
- bridge redirects the customer leg into the configured human queue;
|
|
|
|
|
- when the normal operator-connected flow happens, the same call row becomes `human_owned`.
|
|
|
|
|
|
|
|
|
|
This endpoint is internal-only and trusted for service actors such as `svc:ai-voice-runtime`.
|
|
|
|
|
|
|
|
|
|
## 8.4 `interaction_service`
|
|
|
|
|
|
|
|
|
|
Add one internal endpoint:
|
|
|
|
|
|
|
|
|
|
### `POST /interactions/{interaction_id}/timeline`
|
|
|
|
|
|
|
|
|
|
Request:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"action": "ai.reply_generated",
|
|
|
|
|
"metadata": {
|
|
|
|
|
"call_id": "1740912000.12",
|
|
|
|
|
"voice_session_id": "avs_...",
|
|
|
|
|
"ai_session_id": "ais_..."
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Why add this now:
|
|
|
|
|
|
|
|
|
|
- Voice AI should not write cross-service timeline rows by reaching into another service's DB contract ad hoc;
|
|
|
|
|
- the same endpoint can later be reused by Telegram AI without changing its current behavior immediately;
|
|
|
|
|
- it makes the AI event contract explicit.
|
|
|
|
|
|
|
|
|
|
Required Voice AI timeline actions:
|
|
|
|
|
|
|
|
|
|
- `ai.session_started`
|
|
|
|
|
- `ai.reply_generated`
|
|
|
|
|
- `ai.handoff_requested`
|
|
|
|
|
- `ai.handoff_completed`
|
|
|
|
|
- `ai.error`
|
|
|
|
|
|
|
|
|
|
Existing interaction endpoints remain reused as-is:
|
|
|
|
|
|
|
|
|
|
- `PATCH /interactions/{id}/status`
|
|
|
|
|
- `POST /interactions/{id}/escalate`
|
|
|
|
|
- `PATCH /interactions/{id}/assign`
|
|
|
|
|
|
|
|
|
|
## 8.5 Operator-facing read APIs
|
|
|
|
|
|
|
|
|
|
Extend `asterisk_bridge_service` output:
|
|
|
|
|
|
|
|
|
|
### `GET /asterisk/live-calls`
|
|
|
|
|
|
|
|
|
|
Add to each row:
|
|
|
|
|
|
|
|
|
|
- `voice_session_id`
|
|
|
|
|
- `ai_session_id`
|
|
|
|
|
- `ai_state`
|
|
|
|
|
- `ai_handoff_reason`
|
|
|
|
|
- `ai_last_model_at`
|
|
|
|
|
|
|
|
|
|
### `GET /asterisk/recent-calls`
|
|
|
|
|
|
|
|
|
|
Expose the same additive AI fields.
|
|
|
|
|
|
|
|
|
|
### `GET /asterisk/live-calls/{call_id}/ai-summary`
|
|
|
|
|
|
|
|
|
|
Return a summary shape intentionally aligned with Telegram:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"call_id": "1740912000.12",
|
|
|
|
|
"session_id": "ais_...",
|
|
|
|
|
"voice_session_id": "avs_...",
|
|
|
|
|
"status_label": "AI передал звонок оператору",
|
|
|
|
|
"status_tone": "handoff",
|
|
|
|
|
"customer_request_text": "Клиент хочет узнать статус обращения и изменить способ оплаты",
|
|
|
|
|
"ai_outcome_text": "AI собрал контекст и объяснил рамки, но не выполнил чувствительное действие",
|
|
|
|
|
"handoff_reason": "Запрос требует человека и проверки вручную",
|
|
|
|
|
"recommended_next_step": "Проверить карточку обращения и продолжить звонок вручную",
|
|
|
|
|
"generated_at": "2026-03-09T12:34:56Z"
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This should be produced from `voice_ai_sessions`, `ai_turns`, and the latest transcript segments.
|
|
|
|
|
|
|
|
|
|
## 9. Context Filtering Rules
|
|
|
|
|
|
|
|
|
|
Voice AI must not send the whole call history into the model each turn.
|
|
|
|
|
|
|
|
|
|
For every voice turn, `ai_orchestrator_service` should assemble a filtered context from:
|
|
|
|
|
|
|
|
|
|
- customer profile:
|
|
|
|
|
- `customer_id`
|
|
|
|
|
- display name
|
|
|
|
|
- preferred phone
|
|
|
|
|
- tags
|
|
|
|
|
- customer memory:
|
|
|
|
|
- recent resolved issues
|
|
|
|
|
- notable preferences
|
|
|
|
|
- interaction state:
|
|
|
|
|
- `interaction_id`
|
|
|
|
|
- status
|
|
|
|
|
- queue
|
|
|
|
|
- last relevant timeline events
|
|
|
|
|
- voice session state:
|
|
|
|
|
- language
|
|
|
|
|
- disclosure already played or not
|
|
|
|
|
- previous handoff flag
|
|
|
|
|
- current turn number
|
|
|
|
|
- recent transcript window:
|
|
|
|
|
- last `6-10` finalized segments, not the entire transcript
|
|
|
|
|
- KB:
|
|
|
|
|
- top `3` matched articles or fewer
|
|
|
|
|
- business policy:
|
|
|
|
|
- allowed actions
|
|
|
|
|
- mandatory disclosure
|
|
|
|
|
- sensitive-topic escalation rules
|
|
|
|
|
|
|
|
|
|
This matches the Telegram AI design principle already present in the repo.
|
|
|
|
|
|
|
|
|
|
## 10. Runtime Behavior
|
|
|
|
|
|
|
|
|
|
## 10.1 ASR/TTS abstraction
|
|
|
|
|
|
|
|
|
|
`ai_voice_runtime_service` should expose provider interfaces and start with one configured provider per environment.
|
|
|
|
|
|
|
|
|
|
Recommended internal modules:
|
|
|
|
|
|
|
|
|
|
- `providers/asr.py`
|
|
|
|
|
- `providers/tts.py`
|
|
|
|
|
- `session_manager.py`
|
|
|
|
|
- `media_bridge.py`
|
|
|
|
|
- `barge_in.py`
|
|
|
|
|
|
|
|
|
|
V1 supports one active ASR provider and one active TTS provider, behind interfaces. Multi-provider fallback is not required in the first version.
|
|
|
|
|
|
|
|
|
|
## 10.2 Barge-in
|
|
|
|
|
|
|
|
|
|
Barge-in is a V1 requirement because voice UX breaks if the caller cannot interrupt TTS.
|
|
|
|
|
|
|
|
|
|
Required behavior:
|
|
|
|
|
|
|
|
|
|
- while TTS is playing, incoming speech activity stops or fades out current playback;
|
|
|
|
|
- interrupted assistant output is marked with `barge_in_interrupted=true` in transcript;
|
|
|
|
|
- only finalized caller speech creates a new orchestrator turn;
|
|
|
|
|
- if interruption happens repeatedly or ASR confidence is poor, handoff rules may trigger.
|
|
|
|
|
|
|
|
|
|
## 10.3 Latency budget
|
|
|
|
|
|
|
|
|
|
Target budget for one AI turn:
|
|
|
|
|
|
|
|
|
|
- end-of-utterance to finalized ASR text: `<= 600 ms`
|
|
|
|
|
- orchestrator decision: `<= 900 ms`
|
|
|
|
|
- first TTS audio chunk: `<= 500 ms`
|
|
|
|
|
- total pause before AI speech starts: `<= 2.0 s`
|
|
|
|
|
|
|
|
|
|
If the runtime cannot stay within the budget repeatedly, it should prefer human handoff over a degraded long-silence experience.
|
|
|
|
|
|
|
|
|
|
## 11. Operator UI Integration
|
|
|
|
|
|
|
|
|
|
Voice AI V1 should integrate into the current operator shell, not create a second voice console.
|
|
|
|
|
|
|
|
|
|
### 11.1 Existing surfaces to extend
|
|
|
|
|
|
|
|
|
|
- browser call popup overlay;
|
|
|
|
|
- `Voice debug` live/recent calls list;
|
|
|
|
|
- unified customer history on `/operator`.
|
|
|
|
|
|
|
|
|
|
### 11.2 Required UI changes
|
|
|
|
|
|
|
|
|
|
1. Extend live call rows with an AI chip using the same language as Telegram:
|
|
|
|
|
- `AI active`
|
|
|
|
|
- `Ждёт человека`
|
|
|
|
|
- `AI error`
|
|
|
|
|
2. When a transferred AI-owned call reaches the operator popup:
|
|
|
|
|
- load `GET /asterisk/live-calls/{call_id}/ai-summary`;
|
|
|
|
|
- show a compact `Сводка AI` card above call actions;
|
|
|
|
|
- keep existing `Принять в работу`, `Передать`, `Завершить` controls unchanged.
|
|
|
|
|
3. Add AI metadata to customer history:
|
|
|
|
|
- AI session started
|
|
|
|
|
- AI handoff requested
|
|
|
|
|
- AI handoff completed
|
|
|
|
|
4. Do not add a separate full transcript workspace in V1.
|
|
|
|
|
|
|
|
|
|
### 11.3 UI behavior intentionally deferred
|
|
|
|
|
|
|
|
|
|
Deferred to V2:
|
|
|
|
|
|
|
|
|
|
- operator button `Вернуть AI` for live voice calls;
|
|
|
|
|
- inline live transcript for operators during the call;
|
|
|
|
|
- supervisor transcript explorer for full recordings plus transcripts.
|
|
|
|
|
|
|
|
|
|
## 12. Deployment and Config
|
|
|
|
|
|
|
|
|
|
## 12.1 New service in `docker-compose.server.yml`
|
|
|
|
|
|
|
|
|
|
Add:
|
|
|
|
|
|
|
|
|
|
- `ai-voice-runtime-service`
|
|
|
|
|
|
|
|
|
|
New shared env:
|
|
|
|
|
|
|
|
|
|
- `AI_VOICE_RUNTIME_SERVICE_URL`
|
|
|
|
|
- `AI_VOICE_ENABLED`
|
|
|
|
|
- `AI_VOICE_QUEUE_CONFIG_JSON`
|
|
|
|
|
- `AI_VOICE_ASR_PROVIDER`
|
|
|
|
|
- `AI_VOICE_TTS_PROVIDER`
|
|
|
|
|
- `AI_VOICE_TTS_CACHE_ENABLED`
|
|
|
|
|
- `AI_VOICE_TTS_CACHE_DIR`
|
|
|
|
|
- `AI_VOICE_MAX_CONTEXT_SEGMENTS`
|
|
|
|
|
- `AI_VOICE_HANDOFF_TIMEOUT_SECONDS`
|
|
|
|
|
|
|
|
|
|
Service auth:
|
|
|
|
|
|
|
|
|
|
- add trusted subject `svc:ai-voice-runtime` where internal bridge endpoints require it.
|
|
|
|
|
|
|
|
|
|
## 12.2 Asterisk-side change
|
|
|
|
|
|
|
|
|
|
The accepted human voice path stays intact.
|
|
|
|
|
|
|
|
|
|
Voice AI adds one new AI media bridge path in dialplan only for AI-selected queues. Preferred V1 implementation is:
|
|
|
|
|
|
|
|
|
|
- Asterisk dialplan redirects the customer leg into an external media bridge context dedicated to AI.
|
|
|
|
|
|
|
|
|
|
Exact low-level primitive should be validated in the lab:
|
|
|
|
|
|
|
|
|
|
- preferred: `AudioSocket` or equivalent bidirectional audio bridge;
|
|
|
|
|
- fallback: narrowly scoped external-media/ARI only for AI queues.
|
|
|
|
|
|
|
|
|
|
The choice must keep Asterisk as the telephony owner.
|
|
|
|
|
|
|
|
|
|
## 13. Rollout Order
|
|
|
|
|
|
|
|
|
|
Recommended implementation order:
|
|
|
|
|
|
|
|
|
|
1. schema changes only:
|
|
|
|
|
- `voice_ai_sessions`
|
|
|
|
|
- `voice_transcript_segments`
|
|
|
|
|
- additive AI columns on `asterisk_call_links`
|
|
|
|
|
- additive `call_id` on `ai_sessions`
|
|
|
|
|
2. control plane only:
|
|
|
|
|
- bridge starts and closes empty runtime sessions behind `AI_VOICE_ENABLED=0`
|
|
|
|
|
3. lab media path:
|
|
|
|
|
- one AI-enabled lab queue
|
|
|
|
|
- disclosure + greeting + ASR/TTS echo flow
|
|
|
|
|
4. orchestrator integration:
|
|
|
|
|
- KB lookup
|
|
|
|
|
- filtered context
|
|
|
|
|
- handoff decisioning
|
|
|
|
|
5. operator summary:
|
|
|
|
|
- popup card
|
|
|
|
|
- voice debug AI chips
|
|
|
|
|
6. staged pilot rollout per queue.
|
|
|
|
|
|
|
|
|
|
## 14. Key Risks and Open Questions
|
|
|
|
|
|
|
|
|
|
1. `call_id` stability during AI-to-human redirect must be validated in the Asterisk lab.
|
|
|
|
|
If redirect creates a new call id, bridge recovery must switch to `linked_id` first and only then reuse `interaction_id`.
|
|
|
|
|
2. The exact Asterisk media primitive must be confirmed before implementation.
|
|
|
|
|
The design assumes a bidirectional bridge is available without replacing the accepted telephony baseline.
|
|
|
|
|
3. Provider latency must be measured in the target environment before enabling AI-first for production queues.
|
|
|
|
|
4. Sensitive actions should stay read-only in V1.
|
|
|
|
|
Voice AI should use KB, customer lookup, and safe interaction updates, but not execute risky external business actions directly.
|
|
|
|
|
|
|
|
|
|
## 15. Summary
|
|
|
|
|
|
|
|
|
|
Voice AI V1 should be implemented as an additive AI layer over the accepted voice baseline:
|
|
|
|
|
|
|
|
|
|
- `asterisk_bridge_service` remains telephony truth and handoff executor;
|
|
|
|
|
- `ai_voice_runtime_service` is the new realtime media layer;
|
|
|
|
|
- `ai_orchestrator_service` is reused for deterministic business decisioning;
|
|
|
|
|
- `interaction_service` remains the owner of the canonical AI timeline;
|
|
|
|
|
- operator UI gets AI badges and AI handoff summary, not a new telephony product.
|
|
|
|
|
|
|
|
|
|
This keeps the current live voice contour intact while adding the same AI-first and human-handoff architecture that already works in Telegram.
|
Yera All