docs: recreate documentation structure with updated specs
This commit is contained in:
Yera All
@@ -1,132 +0,0 @@
|
||||
# `.env.production` Checklist (Track 9/11 QA Lab)
|
||||
|
||||
Use this checklist before enabling `ASTERISK_BRIDGE_ENABLED=1`.
|
||||
|
||||
## 1) File handling and secret hygiene
|
||||
|
||||
- [ ] Create local `.env.production` from [`.env.production.template`](/e:/Zhan/.env.production.template).
|
||||
- [ ] Keep `.env.production` out of git (it is ignored in [`.gitignore`](/e:/Zhan/.gitignore)).
|
||||
- [ ] Replace all placeholder values:
|
||||
- `<replace-me>`
|
||||
- `<asterisk-linux-vm-host>`
|
||||
- `<platform_queue_id>`
|
||||
- `<platform_ai_queue_id_or_same_queue_id>`
|
||||
- `<postgres-host>`
|
||||
- `<rabbitmq-host>`
|
||||
- [ ] Verify that AMI/SFTP credentials are not copied into docs, commits, or screenshots.
|
||||
|
||||
## 2) Required environment variables
|
||||
|
||||
### Core
|
||||
|
||||
- [ ] `DATABASE_URL` points to PostgreSQL (not SQLite).
|
||||
- [ ] `INTERACTION_SERVICE_URL` is reachable from `asterisk-bridge-service`.
|
||||
- [ ] `VOICE_ADAPTER_SERVICE_URL` is reachable.
|
||||
- [ ] `RECORDING_SERVICE_URL` is reachable.
|
||||
|
||||
### Track 9 bridge
|
||||
|
||||
- [ ] `ASTERISK_BRIDGE_ENABLED=1`
|
||||
- [ ] `ASTERISK_AMI_HOST` / `ASTERISK_AMI_PORT` are correct.
|
||||
- [ ] `ASTERISK_AMI_USERNAME` / `ASTERISK_AMI_SECRET` are correct.
|
||||
- [ ] `ASTERISK_AMI_EVENT_PREFIX=MVPCC` unless your dialplan intentionally changed it.
|
||||
- [ ] `ASTERISK_QUEUE_MAP_JSON` contains valid JSON object.
|
||||
- [ ] Every `queue_code` in dialplan has a mapped platform `queue_id`.
|
||||
- [ ] `ASTERISK_BRIDGE_AUTH_MODE` chosen intentionally:
|
||||
- `legacy_headers` for Track 9 QA baseline
|
||||
- `bearer_first` for transition hardening
|
||||
- `bearer` for strict service-token mode
|
||||
- [ ] If bearer mode is used, `APP_TOKEN_SECRET` is the same across gateway/services/bridge runtime.
|
||||
- [ ] `ASTERISK_BRIDGE_AUTH_USER` / `ASTERISK_BRIDGE_AUTH_ROLE` match expected service identity.
|
||||
- [ ] `VOICE_ADAPTER_TRUSTED_SERVICE_SUBJECTS` includes `svc:asterisk-bridge`.
|
||||
- [ ] `RECORDING_IMPORT_TRUSTED_SERVICE_SUBJECTS` includes `svc:asterisk-bridge`.
|
||||
- [ ] For strict Track 9.1 mode, set `RECORDING_IMPORT_ALLOW_ADMIN=0`.
|
||||
|
||||
### Track 11 call-control
|
||||
|
||||
- [ ] `ASTERISK_CALLCONTROL_ENABLED=1` (в QA, если проверяете операторское управление звонком).
|
||||
- [ ] `ASTERISK_CALLCONTROL_ACTION_TIMEOUT_SECONDS` задан (рекомендуемо `10`).
|
||||
- [ ] `ASTERISK_OPERATOR_EXTENSION_MAP_JSON` содержит mapping `operator -> extension`.
|
||||
- пример: `{"operator_a":"2001","operator_b":"2002"}`
|
||||
- [ ] `ASTERISK_TRANSFER_TARGET_MAP_JSON` содержит mapping `queue_code -> extension` для blind transfer.
|
||||
- пример: `{"voice_lab":"2001"}`
|
||||
- [ ] `ASTERISK_CALLCONTROL_CLAIM_CONTEXT` и `ASTERISK_CALLCONTROL_TRANSFER_CONTEXT` совпадают с dialplan contexts.
|
||||
|
||||
### SFTP pickup
|
||||
|
||||
- [ ] `ASTERISK_SFTP_HOST` / `ASTERISK_SFTP_PORT` are correct.
|
||||
- [ ] `ASTERISK_SFTP_USERNAME` / `ASTERISK_SFTP_PASSWORD` are correct.
|
||||
- [ ] `ASTERISK_SFTP_BASE_PATH` points to the recording directory (default `/var/spool/asterisk/monitor`).
|
||||
|
||||
### Voice AI V1 staging
|
||||
|
||||
- [ ] `AI_VOICE_ENABLED=1` only for the pilot queues you intend to test.
|
||||
- [ ] `AI_VOICE_QUEUE_CONFIG_JSON` contains the AI-first queue code from the dialplan.
|
||||
- example: `{"voice_lab_ai":{"mode":"ai_first","agent_profile":"voice_support","handoff_queue_code":"voice_lab","language":"ru"}}`
|
||||
- [ ] `AI_VOICE_RUNTIME_SERVICE_URL` points to the HTTP runtime service, not the AudioSocket media endpoint.
|
||||
- [ ] `AI_VOICE_RUNTIME_TRUSTED_SERVICE_SUBJECTS` includes `svc:ai-voice-runtime`.
|
||||
- [ ] `AI_VOICE_ASR_PROVIDER` / `AI_VOICE_TTS_PROVIDER` are set intentionally.
|
||||
- [ ] `ASTERISK_TRANSFER_TARGET_MAP_JSON` includes both the human and AI queue codes when they can hand off to the same operator group.
|
||||
- example: `{"voice_lab":"2001","voice_lab_ai":"2001"}`
|
||||
- [ ] On the Asterisk VM, leave `MVPCC_AI_AUDIOSOCKET_SERVICE=` blank until the media listener is ready.
|
||||
- [ ] After the media listener is available, verify `app_audiosocket.so` is loaded and `MVPCC_AI_AUDIOSOCKET_SERVICE=<media-host>:<port>` uses the raw AudioSocket address.
|
||||
|
||||
### Event bus baseline (Track 8 compatibility)
|
||||
|
||||
- [ ] `EVENT_BUS_ENABLED=1` in QA.
|
||||
- [ ] `EVENT_BUS_URL` resolves to reachable RabbitMQ.
|
||||
|
||||
## 3) Preflight checks before first call
|
||||
|
||||
- [ ] Run migrations:
|
||||
- `python scripts\migrate_core_db.py`
|
||||
- [ ] Run full regression:
|
||||
- `pytest -q`
|
||||
- [ ] Start stack with production env loaded:
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts\start_track9_qa.ps1`
|
||||
- [ ] Bridge health:
|
||||
- `GET /proxy/asterisk-bridge/health` returns `ok`
|
||||
- [ ] Bridge status:
|
||||
- `GET /proxy/asterisk-bridge/asterisk/status` shows `ami_connected=true`
|
||||
|
||||
## 4) Acceptance checks after test call `1001 -> 7000`
|
||||
|
||||
- [ ] `voice` interaction created from Asterisk event flow.
|
||||
- [ ] `voice_events` contain:
|
||||
- `call.started` with `payload.source = "asterisk"`
|
||||
- `call.ended` for the same `call_id`
|
||||
- [ ] `MVPCCRecordingReady` led to uploaded recording in `recording-service`.
|
||||
- [ ] Playback works in `supervisor`.
|
||||
- [ ] Для Track 11:
|
||||
- оператор видит звонок в `/operator` -> блок `Живые звонки`
|
||||
- `claim` работает и пишет action log
|
||||
- `hangup` работает
|
||||
- `blind transfer` работает
|
||||
- [ ] Run:
|
||||
- `python scripts\asterisk_lab_smoke.py --base-url <gateway> --database-url <db> --require-recording`
|
||||
- `python scripts\track9_check.py --base-url <gateway> --database-url <db> --require-recording`
|
||||
- `python scripts\track9_collect_evidence.py --base-url <gateway> --database-url <db> --run-checks --require-recording`
|
||||
|
||||
## 4.1) Voice AI control-plane smoke after test call `1001 -> 7100`
|
||||
|
||||
- [ ] `MVPCCCallStarted` used `QueueCode=voice_lab_ai`.
|
||||
- [ ] `voice_ai_sessions` row created for the call.
|
||||
- [ ] `ai_sessions` row created with `channel=voice`.
|
||||
- [ ] Interaction timeline contains `ai.session_started`.
|
||||
- [ ] Operator UI shows an AI badge on the live or recent voice call card.
|
||||
- [ ] `GET /proxy/asterisk-bridge/asterisk/live-calls/{call_id}/ai-summary` returns `200` once the AI summary exists.
|
||||
- [ ] Human fallback path still works if `MVPCC_AI_AUDIOSOCKET_SERVICE=` is blank.
|
||||
|
||||
## 5) Failure policy
|
||||
|
||||
- [ ] If failed bridge events exist, inspect:
|
||||
- `GET /proxy/asterisk-bridge/asterisk/events?status=failed`
|
||||
- [ ] Retry after fixing root cause:
|
||||
- `POST /proxy/asterisk-bridge/asterisk/events/{bridge_event_id}/retry`
|
||||
- [ ] Do not mark Track 9 accepted until failed bridge events are resolved or explicitly waived.
|
||||
|
||||
## 6) Track 9.2 cutover handoff
|
||||
|
||||
- [ ] Prepare [track9-cutover-sheet.template.md](/e:/Zhan/docs/acceptance/track9/track9-cutover-sheet.template.md).
|
||||
- [ ] Follow [track9-2-production-cutover.md](/e:/Zhan/docs/runbooks/track9-2-production-cutover.md) for controlled Helm rollout.
|
||||
- [ ] Use `scripts\track9_2_cutover.ps1` first in dry-run mode, then with `-Execute` during the approved window.
|
||||
@@ -1,315 +0,0 @@
|
||||
# Asterisk Lab on Linux VM
|
||||
|
||||
Use this runbook to stand up the Track 9 human baseline, the IVR staging entrypoint, and the Voice AI V1 staging entrypoint on the same Asterisk VM.
|
||||
|
||||
## Target
|
||||
|
||||
- Host: dedicated Linux VM
|
||||
- OS: Ubuntu 22.04 LTS or Ubuntu 24.04 LTS
|
||||
- Asterisk: 20 LTS
|
||||
- Integration mode: AMI + custom `UserEvent`
|
||||
- Call source: softphone lab only
|
||||
- Human baseline entrypoint: `7000`
|
||||
- IVR staging entrypoint: `7200`
|
||||
- Voice AI staging entrypoint: `7100`
|
||||
|
||||
## Prepare the VM
|
||||
|
||||
1. Install Asterisk 20 LTS.
|
||||
2. Ensure these ports are reachable from the platform host:
|
||||
- `5060/udp` for SIP
|
||||
- `5038/tcp` for AMI
|
||||
- `22/tcp` for SFTP
|
||||
3. Create the recording directory:
|
||||
- `/var/spool/asterisk/monitor/mvpcc`
|
||||
|
||||
## Apply repo configs
|
||||
|
||||
Copy these files from the repository:
|
||||
|
||||
- `deployment/asterisk/pjsip.conf`
|
||||
- `deployment/asterisk/extensions.conf`
|
||||
- `deployment/asterisk/manager.conf`
|
||||
|
||||
Before reload:
|
||||
|
||||
1. Replace default passwords.
|
||||
2. Confirm extension `1001`.
|
||||
3. Keep extension `7000` as the human-only inbound test entrypoint.
|
||||
4. Keep extension `7100` reserved for Voice AI staging.
|
||||
5. Keep extension `7200` reserved for IVR staging.
|
||||
|
||||
Voice AI dialplan notes:
|
||||
|
||||
- `deployment/asterisk/extensions.conf` now defines two queue codes:
|
||||
- `voice_lab` for `7000`
|
||||
- `voice_lab_ai` for `7100`
|
||||
- `7100` falls back to the human dial targets unless `MVPCC_AI_AUDIOSOCKET_SERVICE` is populated.
|
||||
- When you are ready to test the media bridge, load `app_audiosocket.so` and set `MVPCC_AI_AUDIOSOCKET_SERVICE=<media-host>:<port>`.
|
||||
- Asterisk `AudioSocket()` requires a raw TCP media listener and a UUID per call. The template generates the UUID with `uuidgen`.
|
||||
- Official app reference:
|
||||
- [Asterisk AudioSocket application](https://docs.asterisk.org/Latest_API/API_Documentation/Dialplan_Applications/AudioSocket/)
|
||||
|
||||
IVR dialplan notes:
|
||||
|
||||
- `deployment/asterisk/extensions.conf` defines `voice_lab_ivr` for `7200`.
|
||||
- `7200` calls FastAGI at `MVPCC_IVR_FASTAGI_HOSTPORT` and expects `prompt_audio_key` values to match pre-provisioned Asterisk sound files.
|
||||
- If FastAGI does not return `MVPCC_IVR_TARGET_EXTENSION`, the call falls back to the human dial targets.
|
||||
|
||||
## Configure the platform bridge
|
||||
|
||||
Set these environment variables for `asterisk-bridge-service`:
|
||||
|
||||
- `ASTERISK_BRIDGE_ENABLED=1`
|
||||
- `ASTERISK_AMI_HOST=<linux-vm-host>`
|
||||
- `ASTERISK_AMI_PORT=5038`
|
||||
- `ASTERISK_AMI_USERNAME=mvpcc`
|
||||
- `ASTERISK_AMI_SECRET=<ami-secret>`
|
||||
- `ASTERISK_AMI_EVENT_PREFIX=MVPCC`
|
||||
- `ASTERISK_QUEUE_MAP_JSON={"voice_lab":"<platform_queue_id>","voice_lab_ai":"<platform_ai_queue_id_or_same_queue_id>","voice_lab_ivr":"<platform_ivr_queue_id>"}`
|
||||
- `ASTERISK_BRIDGE_AUTH_MODE=legacy_headers` (or `bearer_first` / `bearer`)
|
||||
- `ASTERISK_BRIDGE_AUTH_FALLBACK_LEGACY=1` (used only with `bearer_first`)
|
||||
- `ASTERISK_BRIDGE_AUTH_SUBJECT=svc:asterisk-bridge`
|
||||
- `ASTERISK_BRIDGE_AUTH_USER=asterisk-bridge`
|
||||
- `ASTERISK_BRIDGE_AUTH_ROLE=admin`
|
||||
- `ASTERISK_BRIDGE_AUTH_TOKEN_TTL_SECONDS=300`
|
||||
- `ASTERISK_IVR_FASTAGI_ENABLED=1`
|
||||
- `ASTERISK_IVR_FASTAGI_HOST=0.0.0.0` (or the explicit bridge bind IP)
|
||||
- `ASTERISK_IVR_FASTAGI_PORT=4573`
|
||||
- `ASTERISK_IVR_DTMF_TIMEOUT_SECONDS=5`
|
||||
- `ASTERISK_IVR_MAX_NO_INPUT_RETRIES=2`
|
||||
- `ASTERISK_IVR_MAX_INVALID_RETRIES=2`
|
||||
- `IVR_RUNTIME_TRUSTED_SERVICE_SUBJECTS=svc:asterisk-bridge`
|
||||
- `VOICE_ADAPTER_TRUSTED_SERVICE_SUBJECTS=svc:asterisk-bridge,svc:ivr-service`
|
||||
- `RECORDING_IMPORT_TRUSTED_SERVICE_SUBJECTS=svc:asterisk-bridge`
|
||||
- `RECORDING_IMPORT_ALLOW_ADMIN=1` (set `0` in strict mode)
|
||||
- `ASTERISK_SFTP_HOST=<linux-vm-host>`
|
||||
- `ASTERISK_SFTP_PORT=22`
|
||||
- `ASTERISK_SFTP_USERNAME=<linux-user>`
|
||||
- `ASTERISK_SFTP_PASSWORD=<linux-password>`
|
||||
- `ASTERISK_SFTP_BASE_PATH=/var/spool/asterisk/monitor`
|
||||
- `ASTERISK_TRANSFER_TARGET_MAP_JSON={"voice_lab":"2001","voice_lab_ai":"2001","voice_lab_ivr":"7200"}`
|
||||
- `AI_VOICE_ENABLED=1`
|
||||
- `AI_VOICE_QUEUE_CONFIG_JSON={"voice_lab_ai":{"mode":"ai_first","agent_profile":"voice_support","handoff_queue_code":"voice_lab","language":"ru"}}`
|
||||
- `AI_VOICE_RUNTIME_SERVICE_URL=http://ai-voice-runtime-service:8000`
|
||||
- `AI_VOICE_RUNTIME_TRUSTED_SERVICE_SUBJECTS=svc:ai-voice-runtime`
|
||||
- `AI_VOICE_ASR_PROVIDER=openai`
|
||||
- `AI_VOICE_TTS_PROVIDER=openai`
|
||||
- `AI_VOICE_MAX_CONTEXT_SEGMENTS=8`
|
||||
- `AI_VOICE_HANDOFF_TIMEOUT_SECONDS=8`
|
||||
|
||||
For Track 9 QA acceptance baseline, keep:
|
||||
|
||||
- bridge auth mode in compatibility headers (`X-User` / `X-Role`)
|
||||
- this is expected for Track 9 baseline and moved to hardening in Track 9.1
|
||||
|
||||
For Track 9.1 hardening, switch to:
|
||||
|
||||
- `ASTERISK_BRIDGE_AUTH_MODE=bearer_first` first
|
||||
- then `ASTERISK_BRIDGE_AUTH_MODE=bearer` after successful smoke and no auth regressions
|
||||
- set `RECORDING_IMPORT_ALLOW_ADMIN=0`
|
||||
- in Helm-based environments, use `deployment/helm/values.track9-strict.yaml` as an overlay
|
||||
|
||||
Voice AI V1 staging guidance:
|
||||
|
||||
- Start with `MVPCC_AI_AUDIOSOCKET_SERVICE=` blank in `extensions.conf`.
|
||||
- This lets `7100` validate the AI control-plane wiring while still falling back to the existing human voice path.
|
||||
- Only point `MVPCC_AI_AUDIOSOCKET_SERVICE` to a live AudioSocket listener after the media listener is reachable from the VM.
|
||||
- `AI_VOICE_RUNTIME_SERVICE_URL` is not the AudioSocket address; it is only the HTTP control-plane URL.
|
||||
|
||||
## Register the softphone
|
||||
|
||||
In your softphone:
|
||||
|
||||
- username: `1001`
|
||||
- password: the value set in `pjsip.conf`
|
||||
- server: the Linux VM IP/hostname
|
||||
|
||||
## QA preflight (required)
|
||||
|
||||
1. Copy [`.env.production.template`](/e:/Zhan/.env.production.template) to local `.env.production`.
|
||||
2. Complete [`.env.production.checklist.md`](/e:/Zhan/docs/runbooks/.env.production.checklist.md).
|
||||
3. Start the stack with production env loaded:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\start_track9_qa.ps1
|
||||
```
|
||||
|
||||
4. Run:
|
||||
|
||||
```powershell
|
||||
python scripts\migrate_core_db.py
|
||||
pytest -q
|
||||
python scripts\track9_preflight.py --base-url http://127.0.0.1:8080 --check-sftp
|
||||
python scripts\track9_preflight.py --base-url http://127.0.0.1:8080 --check-sftp --require-strict-service-auth
|
||||
```
|
||||
|
||||
## Run the first test call
|
||||
|
||||
1. Start the platform.
|
||||
2. Start `asterisk-bridge-service`.
|
||||
3. Call `7000` from extension `1001`.
|
||||
4. Hang up after the playback finishes.
|
||||
|
||||
Expected result:
|
||||
|
||||
- `asterisk-bridge-service` receives:
|
||||
- `MVPCCCallStarted`
|
||||
- `MVPCCCallEnded`
|
||||
- `MVPCCRecordingReady`
|
||||
- platform creates a new `voice` interaction
|
||||
- `voice_events` contain:
|
||||
- `call.started`
|
||||
- `call.ended`
|
||||
- one recording is uploaded into `recording-service`
|
||||
|
||||
## Run the IVR-over-Asterisk smoke
|
||||
|
||||
1. Create or activate an IVR flow for the queue mapped from `voice_lab_ivr`.
|
||||
2. Make sure every live node uses a valid `prompt_audio_key`, and the audio files are present in Asterisk sounds.
|
||||
3. Call `7200` from extension `1001`.
|
||||
4. Test both branches:
|
||||
- enter `1` or `2` to complete IVR and transfer into the mapped queue target
|
||||
- stay silent until retry exhaustion to verify fallback into the human baseline
|
||||
|
||||
Expected result:
|
||||
|
||||
- `asterisk-bridge-service` receives `MVPCCCallStarted` with `QueueCode=voice_lab_ivr`.
|
||||
- `ivr_sessions` contains a live session for the `call_id`.
|
||||
- interaction timeline contains `ivr.session.started` and `ivr.step.no_input` or `ivr.step.completed`.
|
||||
- terminal completion writes `voice_events.event_type="ivr.completed"`.
|
||||
- telephony returns to `mvpcc-transfer` for the resolved queue target, or to the human baseline on retry exhaustion.
|
||||
|
||||
## Run the Voice AI control-plane smoke
|
||||
|
||||
Use this smoke before turning on the real AudioSocket media path.
|
||||
|
||||
1. Keep `MVPCC_AI_AUDIOSOCKET_SERVICE=` blank in `extensions.conf`.
|
||||
2. Start the platform and `ai-voice-runtime-service`.
|
||||
3. Call `7100` from extension `1001`.
|
||||
4. Let the call ring through to the fallback human dial targets.
|
||||
|
||||
Expected result:
|
||||
|
||||
- `asterisk-bridge-service` receives `MVPCCCallStarted` with `QueueCode=voice_lab_ai`.
|
||||
- platform creates:
|
||||
- a `voice` interaction
|
||||
- a `voice_ai_sessions` row
|
||||
- an `ai_sessions` row with `channel=voice`
|
||||
- operator UI shows the AI badge on the live/recent call card
|
||||
- normal call recording flow stays green
|
||||
|
||||
This smoke proves queue selection, bridge-to-runtime session creation, timeline events, and safe human fallback. It does not prove realtime ASR/TTS yet.
|
||||
|
||||
## Verify from the platform
|
||||
|
||||
1. Check:
|
||||
- `GET /proxy/asterisk-bridge/asterisk/status`
|
||||
2. Then run:
|
||||
|
||||
```powershell
|
||||
python scripts\track9_preflight.py --base-url http://127.0.0.1:8080 --check-sftp
|
||||
```
|
||||
|
||||
3. For smoke validation:
|
||||
|
||||
```powershell
|
||||
python scripts\asterisk_lab_smoke.py --base-url http://127.0.0.1:8080 --database-url <database-url> --require-recording
|
||||
```
|
||||
|
||||
4. For formal validation:
|
||||
|
||||
```powershell
|
||||
python scripts\track9_check.py --base-url http://127.0.0.1:8080 --database-url <database-url> --require-recording
|
||||
```
|
||||
|
||||
5. For Voice AI control-plane validation, also verify:
|
||||
- `GET /proxy/asterisk-bridge/asterisk/live-calls`
|
||||
- `GET /proxy/asterisk-bridge/asterisk/live-calls/{call_id}/ai-summary`
|
||||
- `ai.session_started` exists in the interaction timeline for the `7100` call
|
||||
|
||||
## Build formal acceptance evidence pack
|
||||
|
||||
After smoke/check pass:
|
||||
|
||||
```powershell
|
||||
python scripts\track9_collect_evidence.py --base-url http://127.0.0.1:8080 --database-url <database-url> --run-checks --require-recording
|
||||
```
|
||||
|
||||
Then fill:
|
||||
|
||||
- `docs/acceptance/track9/<timestamp>/track9-acceptance.md`
|
||||
|
||||
At minimum include:
|
||||
|
||||
- bridge status snapshot
|
||||
- smoke output
|
||||
- track9_check output
|
||||
- sample rows from:
|
||||
- `asterisk_event_log`
|
||||
- `voice_events`
|
||||
- recording linkage
|
||||
|
||||
## Retry semantics and duplicate safety
|
||||
|
||||
Bridge event statuses:
|
||||
|
||||
- `received`: event accepted by bridge, no forward result yet.
|
||||
- `forwarded`: event processed successfully and forwarded to platform services.
|
||||
- `failed`: forwarding failed; inspect `last_error`, fix root cause, then retry.
|
||||
|
||||
Retry endpoint:
|
||||
|
||||
- `POST /proxy/asterisk-bridge/asterisk/events/{bridge_event_id}/retry`
|
||||
|
||||
Expected behavior:
|
||||
|
||||
1. Retry is safe for the same `bridge_event_id`.
|
||||
2. Existing `call_id -> interaction_id` links prevent duplicate interaction creation.
|
||||
3. Recording retry will update missing upload state if SFTP/upload issue is fixed.
|
||||
4. Keep retries manual in QA until root cause is verified.
|
||||
|
||||
## Timeout / retry policy baseline (QA)
|
||||
|
||||
| Area | Current baseline | Operator action |
|
||||
|---|---|---|
|
||||
| AMI reconnect | `ASTERISK_BRIDGE_POLL_INTERVAL_SECONDS` (default `1s`) reconnect loop | Verify `ami_connected=true` in `/asterisk/status` |
|
||||
| SFTP fetch | Single attempt per event processing | Fix credentials/path/network, then call retry endpoint |
|
||||
| Recording upload | Single upload attempt per event processing | Fix `recording-service` reachability/auth, then retry |
|
||||
| Failed event aging | No auto-waiver | Track 9 is not accepted with unresolved failed events |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- `AMI is not connected`
|
||||
- verify `manager.conf`, firewall, and credentials
|
||||
- `Unknown QueueCode`
|
||||
- ensure dialplan `QueueCode` matches `ASTERISK_QUEUE_MAP_JSON`
|
||||
- for Voice AI staging, verify both `voice_lab` and `voice_lab_ai`
|
||||
- verify mapped `queue_id` exists in platform:
|
||||
- `GET /proxy/routing/queues`
|
||||
- re-run failed event with retry endpoint
|
||||
- `7100` immediately falls back to a human and no AI badge appears
|
||||
- verify `AI_VOICE_ENABLED=1`
|
||||
- verify `AI_VOICE_QUEUE_CONFIG_JSON` contains `voice_lab_ai`
|
||||
- verify the `MVPCCCallStarted` event shows `QueueCode=voice_lab_ai`
|
||||
- `7100` errors as soon as AudioSocket is enabled
|
||||
- verify `app_audiosocket.so` is loaded on Asterisk
|
||||
- verify `MVPCC_AI_AUDIOSOCKET_SERVICE` points to a raw TCP media listener, not `http://...:8000`
|
||||
- verify the media host and port are reachable from the Asterisk VM
|
||||
- `No uploaded recordings linked to Asterisk events`
|
||||
- verify SFTP credentials and `/var/spool/asterisk/monitor/mvpcc`
|
||||
- verify `ASTERISK_SFTP_BASE_PATH`
|
||||
- verify `POST /proxy/recording/recordings/import-upload` path is reachable from bridge host
|
||||
- retry failed `MVPCCRecordingReady` events after fix
|
||||
- `Failed bridge events present`
|
||||
- inspect:
|
||||
- `GET /proxy/asterisk-bridge/asterisk/events?status=failed`
|
||||
- then retry with:
|
||||
- `POST /proxy/asterisk-bridge/asterisk/events/{bridge_event_id}/retry`
|
||||
|
||||
## Track 9.2 production cutover
|
||||
|
||||
After QA acceptance is signed, switch to controlled production rollout:
|
||||
|
||||
- [track9-2-production-cutover.md](/e:/Zhan/docs/runbooks/track9-2-production-cutover.md)
|
||||
- `scripts/track9_2_cutover.ps1`
|
||||
@@ -1,20 +0,0 @@
|
||||
# Runbook - Backup and Restore (Stage 4)
|
||||
|
||||
## Backup
|
||||
- Stop write-heavy operations (or enter maintenance window).
|
||||
- Run backup script:
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/backup_data.ps1`
|
||||
- Keep backup config (`values.yaml`, contracts, runbooks).
|
||||
|
||||
## Restore drill
|
||||
1. Start clean environment
|
||||
2. Re-apply deployment manifests
|
||||
3. Restore snapshots:
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts/restore_data.ps1 -BackupZip <zip-path>`
|
||||
4. Validate smoke scenarios:
|
||||
- login
|
||||
- create interaction
|
||||
- KPI endpoint
|
||||
|
||||
## Acceptance
|
||||
- RTO and data integrity documented in pilot protocol
|
||||
@@ -1,79 +0,0 @@
|
||||
# Demo Showcase Runbook
|
||||
|
||||
Use this flow before showing the MVP to management.
|
||||
|
||||
## One-command prep
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\prepare_demo.ps1
|
||||
```
|
||||
|
||||
What it does:
|
||||
|
||||
- starts the full local stack in the background
|
||||
- clears local demo data before start
|
||||
- runs the smoke check against `http://localhost:8080`
|
||||
- seeds demo data for customers, interactions, integrations, KB, supervisor, KPI, and one sample recording
|
||||
|
||||
## What to open
|
||||
|
||||
- Operator UI: `http://localhost:8080/operator`
|
||||
- Supervisor UI: `http://localhost:8080/supervisor`
|
||||
- Admin UI: `http://localhost:8080/admin`
|
||||
- Logs: `.local_stack\logs`
|
||||
- Demo summary: `.local_stack\demo-seed-summary.json`
|
||||
- Login: `admin / admin123`
|
||||
|
||||
## Demo sequence
|
||||
|
||||
1. Log in as `admin / admin123`.
|
||||
2. Open the interactions list and show:
|
||||
- one voice interaction already assigned, escalated, and closed
|
||||
- one active follow-up interaction
|
||||
3. Show that assign / escalate uses default values (`operator_a`, `line2`) without browser prompts.
|
||||
4. Trigger the voice channel, Telegram, Webchat, and Email checks.
|
||||
5. Search KB for `demo-showcase`.
|
||||
6. Refresh the supervisor block and read the summary cards.
|
||||
7. Refresh KPI and read the summary cards.
|
||||
- the JSON payload also includes extended metrics and a `by_channel` breakdown
|
||||
8. Open `/supervisor`, load `Recordings`, and show playback/download of the seeded sample.
|
||||
9. Open `/admin`, load the seeded IVR flow, show the completed IVR session, and preview route override with `ivr_session_id`.
|
||||
|
||||
## Optional Telegram chat follow-up
|
||||
|
||||
If the Telegram bot env is configured:
|
||||
|
||||
1. Open `/operator`.
|
||||
2. Go to the `Telegram` page in the sidebar.
|
||||
3. Send one test message to the configured bot from a real Telegram client.
|
||||
4. Show that:
|
||||
- a persistent thread appears for that `chat_id`
|
||||
- the linked interaction uses `channel="telegram"`
|
||||
- the operator can claim the thread and reply from the same page
|
||||
|
||||
## Optional live voice follow-up
|
||||
|
||||
If the Asterisk lab bridge is enabled on a dedicated Linux VM:
|
||||
|
||||
1. Open `/operator`.
|
||||
2. Choose the QA media path:
|
||||
- browser softphone via `Browser Softphone`
|
||||
- or external fallback via `MicroSIP/Zoiper`
|
||||
3. Place one lab call from softphone `1001` to `7000`.
|
||||
4. If using browser media, connect the browser softphone first and answer in Chrome/Edge.
|
||||
5. Use the current voice baseline from `docs/runbooks/voice-baseline.md`.
|
||||
|
||||
## Optional separate shells
|
||||
|
||||
After the main management flow, you can open:
|
||||
|
||||
- `/supervisor` to show dedicated realtime and queue controls
|
||||
- `/admin` to show separate user and queue management
|
||||
|
||||
## Stop after the demo
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\stop_all_local.ps1
|
||||
```
|
||||
|
||||
For a short speaking script during the meeting, use `docs/runbooks/management-demo-brief.md`.
|
||||
@@ -1,175 +0,0 @@
|
||||
# Runbook - On-Prem Deployment
|
||||
|
||||
## Helm
|
||||
- `helm upgrade --install mvp-cc ./deployment/helm -n mvp-cc --create-namespace`
|
||||
- For PostgreSQL-backed Helm releases, the chart now runs `scripts/migrate_core_db.py` automatically via a pre-install / pre-upgrade migration Job.
|
||||
|
||||
## Raw manifests
|
||||
- `kubectl apply -f deployment/kubernetes/mvp-cc-platform.yaml`
|
||||
- Raw manifests remain a minimal reference/bootstrap path.
|
||||
- Helm is the canonical production-scale deployment path for Wave 2 / Track 7.
|
||||
|
||||
## Core DB migration
|
||||
- Helm path:
|
||||
- keep `migrations.enabled=true`
|
||||
- keep `SCHEMA_MANAGEMENT_MODE=migrations`
|
||||
- the Helm hook Job applies migrations before rollout
|
||||
- Raw manifests / manual path:
|
||||
- run `python scripts/migrate_core_db.py`
|
||||
- then start services with `SCHEMA_MANAGEMENT_MODE=migrations`
|
||||
|
||||
## Wave 2 / Track 7 scale baseline
|
||||
|
||||
Production-like K8s scale validation requires:
|
||||
|
||||
- `DATABASE_URL` set to a shared PostgreSQL endpoint
|
||||
- `DB_POOL_SIZE`
|
||||
- `DB_MAX_OVERFLOW`
|
||||
- `DB_POOL_TIMEOUT_SECONDS`
|
||||
- `DB_POOL_RECYCLE_SECONDS`
|
||||
- a working `metrics-server` in the target cluster
|
||||
|
||||
Recommended Wave 2 / Track 7 rollout:
|
||||
|
||||
1. Render Helm config for the scale profile.
|
||||
2. Confirm `DATABASE_URL` points to PostgreSQL (not SQLite).
|
||||
3. Confirm `SCHEMA_MANAGEMENT_MODE=migrations`.
|
||||
4. Confirm `migrations.enabled=true`.
|
||||
5. Deploy via Helm.
|
||||
6. Verify the migration Job completed successfully.
|
||||
7. Run `python scripts/load_test.py --base-url http://<gateway-host> --profile step_250_250 --auth-mode bearer`.
|
||||
8. Run `python scripts/track7_check.py --namespace mvp-cc --report-dir <report_dir>`.
|
||||
9. Repeat with `--profile target_500_500`.
|
||||
|
||||
For the complete procedure, use:
|
||||
|
||||
- [track7-scale-validation.md](/e:/Zhan/docs/runbooks/track7-scale-validation.md)
|
||||
|
||||
## Wave 2 / Track 1 auth configuration
|
||||
|
||||
Minimum enterprise auth variables:
|
||||
|
||||
- `APP_TOKEN_SECRET`
|
||||
- `APP_TOKEN_TTL_SECONDS` (default `3600`)
|
||||
- `ALLOW_LEGACY_HEADER_AUTH=0`
|
||||
- `OIDC_ENABLED=1`
|
||||
- `OIDC_PROVIDER=keycloak`
|
||||
- `OIDC_ISSUER_URL`
|
||||
- `OIDC_CLIENT_ID`
|
||||
- `OIDC_CLIENT_SECRET`
|
||||
- `OIDC_REDIRECT_URI`
|
||||
- `OIDC_SCOPES` (default `openid profile email`)
|
||||
- `OIDC_ROLE_CLAIM` (default `groups`)
|
||||
- `OIDC_ROLE_MAP_JSON`
|
||||
|
||||
Helm and raw Kubernetes manifests now include placeholder values for these settings.
|
||||
Replace all `change-me-*` values before deploying outside local/demo.
|
||||
|
||||
Recommended rollout sequence:
|
||||
|
||||
1. For Helm, keep migration hook enabled so schema upgrades run before deploy.
|
||||
2. Deploy with `OIDC_ENABLED=1` and `ALLOW_LEGACY_HEADER_AUTH=1` in QA.
|
||||
3. Run `python scripts/oidc_smoke.py --base-url http://<gateway-host> --require-enabled`.
|
||||
4. Validate a real Keycloak login round trip.
|
||||
5. Switch enterprise pilot to `ALLOW_LEGACY_HEADER_AUTH=0`.
|
||||
|
||||
## Post-deploy checks
|
||||
- Pods ready state
|
||||
- `/health` probes green
|
||||
- Gateway registry endpoint available
|
||||
- `GET /proxy/auth/auth/oidc/config` returns `enabled=true`
|
||||
- `GET /proxy/auth/auth/oidc/health` returns provider metadata status
|
||||
|
||||
See also:
|
||||
|
||||
- [keycloak-oidc.md](/e:/Zhan/docs/runbooks/keycloak-oidc.md)
|
||||
- [postgres-dev-cutover.md](/e:/Zhan/docs/runbooks/postgres-dev-cutover.md)
|
||||
- [postgres-server-docker.md](/e:/Zhan/docs/runbooks/postgres-server-docker.md)
|
||||
- [postgres-server-parallel-stack.md](/e:/Zhan/docs/runbooks/postgres-server-parallel-stack.md)
|
||||
|
||||
## Wave 2 / Track 9.1 strict bridge auth cutover
|
||||
|
||||
Use Helm overlays to force strict bridge auth mode in QA/enterprise environments:
|
||||
|
||||
```powershell
|
||||
helm upgrade --install mvp-cc deployment\helm `
|
||||
-f deployment\helm\values.scale500.yaml `
|
||||
-f deployment\helm\values.track9-strict.yaml `
|
||||
--set-string auth.appTokenSecret=<shared-app-token-secret> `
|
||||
--set-string asteriskBridge.enabled=1 `
|
||||
--set-string asteriskBridge.amiHost=<asterisk-vm-host> `
|
||||
--set-string asteriskBridge.amiUsername=<ami-user> `
|
||||
--set-string asteriskBridge.amiSecret=<ami-secret> `
|
||||
--set-string asteriskBridge.queueMapJson='{\"voice_lab\":\"<queue_id>\"}' `
|
||||
--set-string asteriskBridge.sftpHost=<asterisk-vm-host> `
|
||||
--set-string asteriskBridge.sftpUsername=<sftp-user> `
|
||||
--set-string asteriskBridge.sftpPassword=<sftp-password> `
|
||||
-n mvp-cc --create-namespace
|
||||
```
|
||||
|
||||
Expected strict baseline after deploy:
|
||||
|
||||
- `ALLOW_LEGACY_HEADER_AUTH=0`
|
||||
- `ASTERISK_BRIDGE_AUTH_MODE=bearer`
|
||||
- `ASTERISK_BRIDGE_AUTH_FALLBACK_LEGACY=0`
|
||||
- `VOICE_ADAPTER_TRUSTED_SERVICE_SUBJECTS` contains `svc:asterisk-bridge`
|
||||
- `RECORDING_IMPORT_TRUSTED_SERVICE_SUBJECTS` contains `svc:asterisk-bridge`
|
||||
- `RECORDING_IMPORT_ALLOW_ADMIN=0`
|
||||
|
||||
Post-cutover validation:
|
||||
|
||||
```powershell
|
||||
python scripts\track9_preflight.py --base-url http://<gateway-host> --check-sftp
|
||||
python scripts\asterisk_lab_smoke.py --base-url http://<gateway-host> --database-url <database-url> --require-recording
|
||||
python scripts\track9_check.py --base-url http://<gateway-host> --database-url <database-url> --require-recording
|
||||
```
|
||||
|
||||
## Wave 2 / Track 9.2 controlled production cutover
|
||||
|
||||
Use the dedicated cutover runbook and script:
|
||||
|
||||
- [track9-2-production-cutover.md](/e:/Zhan/docs/runbooks/track9-2-production-cutover.md)
|
||||
- `scripts/track9_2_cutover.ps1`
|
||||
|
||||
Dry-run (render/lint/snapshot only):
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\track9_2_cutover.ps1 `
|
||||
-KubeContext <context> `
|
||||
-Namespace <namespace> `
|
||||
-Release <release> `
|
||||
-GatewayBaseUrl http://<gateway-host> `
|
||||
-DatabaseUrl postgresql://<...> `
|
||||
-ImageTag <image-tag> `
|
||||
-AmiHost <asterisk-vm-host> `
|
||||
-AmiUser <ami-user> `
|
||||
-AmiSecret <ami-secret> `
|
||||
-SftpHost <asterisk-vm-host> `
|
||||
-SftpUser <sftp-user> `
|
||||
-SftpPassword <sftp-password> `
|
||||
-QueueId <voice-lab-queue-id> `
|
||||
-AppTokenSecret <shared-app-token-secret>
|
||||
```
|
||||
|
||||
Execute cutover:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\track9_2_cutover.ps1 `
|
||||
-KubeContext <context> `
|
||||
-Namespace <namespace> `
|
||||
-Release <release> `
|
||||
-GatewayBaseUrl http://<gateway-host> `
|
||||
-DatabaseUrl postgresql://<...> `
|
||||
-ImageTag <image-tag> `
|
||||
-AmiHost <asterisk-vm-host> `
|
||||
-AmiUser <ami-user> `
|
||||
-AmiSecret <ami-secret> `
|
||||
-SftpHost <asterisk-vm-host> `
|
||||
-SftpUser <sftp-user> `
|
||||
-SftpPassword <sftp-password> `
|
||||
-QueueId <voice-lab-queue-id> `
|
||||
-AppTokenSecret <shared-app-token-secret> `
|
||||
-Execute
|
||||
```
|
||||
|
||||
If Helm reports HPA scale field conflicts, rerun the same command with `-ForceUpgrade`.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Руководство по развертыванию (Deployment)
|
||||
|
||||
Контакт-центр поддерживает два основных сценария развертывания на серверах: упрощенный (через Docker Compose для небольших стендов) и enterprise-grade (через Kubernetes и Helm для продакшена).
|
||||
|
||||
## 1. Требования к Production-окружению
|
||||
|
||||
Для запуска в боевых условиях (Scale Profile) **строго обязателен PostgreSQL**. Запуск на SQLite в Production категорически не поддерживается.
|
||||
Также для связи микросервисов в продакшене требуется запущенный кластер **RabbitMQ**.
|
||||
|
||||
## 2. Развертывание через серверный Docker Compose (Test / Stage)
|
||||
|
||||
Если у вас один выделенный сервер (VPS/VM Linux) и вы хотите настроить автоматический деплой из GitLab:
|
||||
|
||||
Готовые скрипты находятся в `scripts/`:
|
||||
- `install_gitlab_runner.sh` — регистрирует раннер на сервере.
|
||||
- `bootstrap_gitlab_deploy.sh` — подготавливает базовые папки и .env-файлы.
|
||||
- `deploy_gitlab.sh` — сам скрипт деплоя.
|
||||
|
||||
В директории `deployment/` лежат файлы конфигураций для сервера:
|
||||
- `docker-compose.server.registry.yml` — тянет собранные образы напрямую из GitLab Container Registry.
|
||||
- `.env.production.template` — шаблон боевых секретов, включая `APP_TOKEN_SECRET` и `OIDC_...` для Keycloak.
|
||||
|
||||
**Как обновиться (CI/CD Pipeline):**
|
||||
GitLab CI (файл `.gitlab-ci.yml`) собирает Docker-образы для каждого микросервиса (`services/*`) при merge в ветку `main` и перезапускает Compose-файл на вашем выделенном сервере.
|
||||
|
||||
## 3. Развертывание в Kubernetes (On-premise / Cloud Production)
|
||||
|
||||
Для крупных внедрений контакт-центра (когда нужен автомасштаб `ai_voice_runtime_service` или `routing-service` при высокой нагрузке) используется встроенный Helm-чарт.
|
||||
|
||||
Исходники манифестов лежат в:
|
||||
- `deployment/helm/` — основной Helm Chart микросервисной платформы.
|
||||
- `deployment/kubernetes/` — статические манифесты.
|
||||
|
||||
### 3.1. Установка Helm чарта
|
||||
Для установки чарта в кластер перейдите в директорию `deployment/helm` и выполните:
|
||||
```bash
|
||||
# 1. Создание отдельного namespace
|
||||
kubectl create namespace mvp-cc
|
||||
|
||||
# 2. Установка/обновление платформы
|
||||
helm upgrade --install mvp-cc-prod ./ \
|
||||
--namespace mvp-cc \
|
||||
-f values.yaml \
|
||||
-f values.prod.yaml
|
||||
```
|
||||
|
||||
### 3.2. Масштабирование (Scale Profiles)
|
||||
Шаблоны Helm поддерживают включение профилей нагрузки. Например, для запуска `scale500` (профиль на 500 одновременных линий/пользователей):
|
||||
- Отключите встроенные SQLite-заглушки во `values.yaml`.
|
||||
- Строго пропишите внешний `DATABASE_URL` до высокодоступного кластера PostgreSQL.
|
||||
- Настройте пулинг коннектов:
|
||||
- `DB_POOL_SIZE`
|
||||
- `DB_MAX_OVERFLOW`
|
||||
|
||||
## 4. Резервное копирование и Откат конфигураций
|
||||
|
||||
Даже при работе в Kubernetes базу данных (PostgreSQL) желательно держать вне кластера (Managed Database/Patroni).
|
||||
|
||||
**Бэкапы аудиозаписей:**
|
||||
Если вы используете `recording-service` с локальным хранением, смонтированные PVC (Persistent Volume Claims) с аудиозаписями нужно регулярно копировать. В инфраструктуре без K8s для этого предусмотрен скрипт `scripts/backup_data.ps1`.
|
||||
@@ -1,83 +0,0 @@
|
||||
# Runbook - Event Bus (Track 8, Local)
|
||||
|
||||
## Goal
|
||||
|
||||
Run the first RabbitMQ-backed event bus flow locally and verify:
|
||||
|
||||
- outbox writes happen in business services
|
||||
- `event-bus-service` publishes events
|
||||
- `audit-service` consumes them
|
||||
- `reporting-service` consumes them
|
||||
|
||||
## Local compose path
|
||||
|
||||
Use Docker Compose because it includes RabbitMQ:
|
||||
|
||||
```powershell
|
||||
cd deployment
|
||||
docker compose up -d
|
||||
cd ..
|
||||
python scripts\migrate_core_db.py
|
||||
```
|
||||
|
||||
RabbitMQ management UI:
|
||||
|
||||
- `http://localhost:15672`
|
||||
- default local credentials: `guest / guest`
|
||||
|
||||
## Required env
|
||||
|
||||
The compose profile already sets:
|
||||
|
||||
- `EVENT_BUS_ENABLED=1`
|
||||
- `EVENT_BUS_URL=amqp://guest:guest@rabbitmq:5672/`
|
||||
- `EVENT_BUS_CONSUMER_ENABLED=1`
|
||||
|
||||
For non-compose local runs, export them manually before starting services.
|
||||
|
||||
## Smoke check
|
||||
|
||||
```powershell
|
||||
python scripts\event_bus_smoke.py --base-url http://localhost:8080
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- gateway and `event-bus-service` are healthy
|
||||
- an interaction is created
|
||||
- its outbox event becomes `published`
|
||||
- `audit-service` processes the event
|
||||
- `reporting-service` processes the event
|
||||
|
||||
## Inspect outbox
|
||||
|
||||
```powershell
|
||||
curl http://localhost:8080/proxy/event-bus/bus/outbox -H "X-User: admin" -H "X-Role: admin"
|
||||
```
|
||||
|
||||
Useful statuses:
|
||||
|
||||
- `pending`
|
||||
- `published`
|
||||
- `failed`
|
||||
|
||||
## Retry a failed event
|
||||
|
||||
```powershell
|
||||
curl -X POST http://localhost:8080/proxy/event-bus/bus/outbox/<event_id>/retry -H "X-User: admin" -H "X-Role: admin"
|
||||
```
|
||||
|
||||
## Acceptance check
|
||||
|
||||
```powershell
|
||||
python scripts\track8_check.py --base-url http://localhost:8080
|
||||
```
|
||||
|
||||
## Compatibility mode
|
||||
|
||||
If you want to disable the bus and keep the platform on pure HTTP-only behavior:
|
||||
|
||||
- set `EVENT_BUS_ENABLED=0`
|
||||
- restart services
|
||||
|
||||
The business REST APIs keep working in that mode.
|
||||
@@ -1,80 +0,0 @@
|
||||
# GitLab CI/CD for call-center
|
||||
|
||||
## What this setup does
|
||||
|
||||
- a push to `main` triggers GitLab CI
|
||||
- the job runs on a dedicated `shell` runner with tag `call-center-prod`
|
||||
- the runner syncs the repository into `/home/gitlab-runner/deploy/call-center`
|
||||
- Docker builds `call-center-app:<commit-sha>` and also tags `call-center-app:latest`
|
||||
- `docker compose` recreates the stack from `deployment/docker-compose.server.yml`
|
||||
- the public gateway container is named `call-center-app`
|
||||
|
||||
## Required server prerequisites
|
||||
|
||||
- Docker installed and running
|
||||
- outbound access to `https://gitlab.konturai.kz`
|
||||
- runner token from GitLab with prefix `glrt-...` or a valid project/group runner token
|
||||
|
||||
## Install and register the runner
|
||||
|
||||
Run on the target server as `root`:
|
||||
|
||||
```bash
|
||||
cd /path/to/call-center
|
||||
RUNNER_TOKEN=glrt-xxxxxxxx bash scripts/install_gitlab_runner.sh
|
||||
```
|
||||
|
||||
The script:
|
||||
|
||||
- installs `gitlab-runner` from the official GitLab repository
|
||||
- adds user `gitlab-runner` to the `docker` group
|
||||
- creates deploy directory `/home/gitlab-runner/deploy/call-center`
|
||||
- registers runner `call-center-prod-runner` with tag `call-center-prod`
|
||||
|
||||
If you first want to install the service without registration:
|
||||
|
||||
```bash
|
||||
SKIP_REGISTER=1 bash scripts/install_gitlab_runner.sh
|
||||
```
|
||||
|
||||
## Production environment file
|
||||
|
||||
Create the production env file once on the server:
|
||||
|
||||
```bash
|
||||
install -m 600 /dev/null /home/gitlab-runner/deploy/call-center/.env.production
|
||||
```
|
||||
|
||||
Then fill it with the values required by `deployment/docker-compose.server.yml`.
|
||||
|
||||
If the project is already running from `/root/call-center`, migrate the current env file and SQLite/files before the first CI deploy:
|
||||
|
||||
```bash
|
||||
bash scripts/bootstrap_gitlab_deploy.sh
|
||||
```
|
||||
|
||||
Alternative:
|
||||
|
||||
- keep the env file elsewhere
|
||||
- pass `DEPLOY_ENV_FILE=/absolute/path/to/.env.production` in GitLab CI/CD variables
|
||||
|
||||
## GitLab CI/CD variables
|
||||
|
||||
Optional project variables:
|
||||
|
||||
- `DEPLOY_DIR` if you want a different deploy directory
|
||||
- `APP_IMAGE_NAME` if you want a different Docker image name
|
||||
- `HEALTHCHECK_URL` if the gateway health URL differs
|
||||
- `DEPLOY_ENV_FILE` if `.env.production` should be copied from another location
|
||||
|
||||
## First deployment
|
||||
|
||||
1. Register the runner.
|
||||
2. Add the production env file.
|
||||
3. Push this configuration to the `main` branch.
|
||||
4. Confirm the pipeline completes successfully.
|
||||
5. Verify the container:
|
||||
|
||||
```bash
|
||||
docker ps --filter name=call-center-app
|
||||
```
|
||||
@@ -1,86 +0,0 @@
|
||||
# GitLab -> Server Deploy
|
||||
|
||||
This project is prepared for a GitLab-first delivery flow:
|
||||
|
||||
1. Push source code to GitLab.
|
||||
2. GitLab CI runs tests.
|
||||
3. GitLab CI builds and pushes container images to GitLab Container Registry.
|
||||
4. The server only receives the deployment bundle and pulls images from the registry.
|
||||
|
||||
For a same-host shell-runner flow that rebuilds `call-center-app` directly on push to `main`, see:
|
||||
|
||||
- `docs/runbooks/gitlab-cicd.md`
|
||||
|
||||
## Files used
|
||||
|
||||
- `.gitlab-ci.yml`
|
||||
- `deployment/docker-compose.server.registry.yml`
|
||||
- `deployment/docker-compose.asterisk.server.registry.yml`
|
||||
- `deployment/.env.images.example`
|
||||
|
||||
## GitLab CI variables
|
||||
|
||||
Required for image publishing:
|
||||
|
||||
- `CI_REGISTRY`
|
||||
- `CI_REGISTRY_USER`
|
||||
- `CI_REGISTRY_PASSWORD`
|
||||
- `CI_REGISTRY_IMAGE`
|
||||
|
||||
Required for the optional deploy job:
|
||||
|
||||
- `DEPLOY_HOST`
|
||||
- `DEPLOY_USER`
|
||||
- `DEPLOY_PATH`
|
||||
- `DEPLOY_SSH_PRIVATE_KEY`
|
||||
|
||||
`DEPLOY_PATH` should point to a minimal runtime directory on the server, for example:
|
||||
|
||||
```text
|
||||
/opt/call-center
|
||||
```
|
||||
|
||||
The server runtime directory should contain:
|
||||
|
||||
- `.env.production`
|
||||
- `.data_local/`
|
||||
- `.asterisk_assets/` (can stay empty)
|
||||
- `deployment/`
|
||||
|
||||
## Server deploy bundle
|
||||
|
||||
The server no longer needs the full project checkout. It only needs:
|
||||
|
||||
- `deployment/`
|
||||
- `.env.production`
|
||||
- `.data_local/`
|
||||
- `.asterisk_assets/` if you use custom prompt files
|
||||
|
||||
The app image is resolved through `APP_IMAGE`.
|
||||
The Asterisk image is resolved through `ASTERISK_IMAGE`.
|
||||
|
||||
Store them in `deployment/.env.images`, for example:
|
||||
|
||||
```text
|
||||
APP_IMAGE=registry.gitlab.example.com/group/project/app:<tag>
|
||||
ASTERISK_IMAGE=registry.gitlab.example.com/group/project/asterisk:<tag>
|
||||
```
|
||||
|
||||
## Manual server deploy
|
||||
|
||||
From the server:
|
||||
|
||||
```bash
|
||||
cd /opt/call-center/deployment
|
||||
docker login <registry>
|
||||
docker compose --env-file .env.images -f docker-compose.server.registry.yml pull
|
||||
docker compose --env-file .env.images -f docker-compose.server.registry.yml up -d
|
||||
docker compose --env-file .env.images -f docker-compose.asterisk.server.registry.yml pull
|
||||
docker compose --env-file .env.images -f docker-compose.asterisk.server.registry.yml up -d
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- `deployment/docker-compose.server.yml` is still the source-build variant.
|
||||
- `deployment/docker-compose.server.registry.yml` is the registry/pull-only variant for the server.
|
||||
- The deploy job in `.gitlab-ci.yml` is manual on the default branch by design.
|
||||
@@ -1,102 +0,0 @@
|
||||
# IVR Local Runbook
|
||||
|
||||
Use this runbook to validate both the preview IVR runtime and the live Asterisk-backed IVR staging path.
|
||||
|
||||
## What Track 5 adds
|
||||
|
||||
- `ivr-service` for IVR flow CRUD and DTMF runtime sessions
|
||||
- one active IVR flow per voice queue
|
||||
- `routing-service` support for `ivr_session_id`
|
||||
- admin-side IVR preview flow in `/admin`
|
||||
|
||||
## Preview baseline vs live IVR
|
||||
|
||||
- Preview baseline: use the `/admin` IVR block to create flows, start sessions, send DTMF, and preview route overrides without telephony.
|
||||
- Live Asterisk IVR: call `7200` on the lab PBX after `asterisk-bridge-service` is configured with `ASTERISK_IVR_FASTAGI_*` and the flow nodes have valid `prompt_audio_key` values.
|
||||
|
||||
## Create a flow
|
||||
|
||||
1. Open `http://localhost:8080/admin`
|
||||
2. Go to the `IVR` block
|
||||
3. Set:
|
||||
- `queue_id`
|
||||
- `entry_node_id` (`root`)
|
||||
- `flow_json`
|
||||
4. Click `Create flow`
|
||||
|
||||
The flow JSON is validated before it is saved.
|
||||
|
||||
## Start a session
|
||||
|
||||
1. In the same `IVR` block, set:
|
||||
- `call_id`
|
||||
- `queue_id`
|
||||
- optional `interaction_id`
|
||||
2. Click `Start session`
|
||||
|
||||
The response returns:
|
||||
|
||||
- `session_id`
|
||||
- `current_node_id`
|
||||
- current node prompt metadata
|
||||
|
||||
## Send DTMF
|
||||
|
||||
1. Enter a single digit (`0`-`9`)
|
||||
2. Click `Send DTMF`
|
||||
|
||||
If the transition reaches a terminal node:
|
||||
|
||||
- the session becomes `completed`
|
||||
- `outcome_code` is set
|
||||
- `resolved_queue_id` is set
|
||||
- `resolved_queue_code` is set
|
||||
- a `voice_event` with `event_type="ivr.completed"` is written
|
||||
|
||||
## Preview route override
|
||||
|
||||
1. Keep the completed `session_id`
|
||||
2. Click `Preview IVR route`
|
||||
|
||||
This calls `routing-service` with `ivr_session_id` and shows:
|
||||
|
||||
- `original_queue_id`
|
||||
- `resolved_queue_id`
|
||||
- `ivr_outcome_code`
|
||||
|
||||
## Demo seed
|
||||
|
||||
`scripts/prepare_demo.ps1` now seeds:
|
||||
|
||||
- one demo IVR flow
|
||||
- one completed demo IVR session
|
||||
- one route preview stored in `.local_stack/demo-seed-summary.json`
|
||||
|
||||
Backstage fields:
|
||||
|
||||
- `ivr_flow_id`
|
||||
- `ivr_session_id`
|
||||
- `ivr_outcome_code`
|
||||
- `ivr_resolved_queue_id`
|
||||
|
||||
## Live Asterisk smoke
|
||||
|
||||
1. Enable:
|
||||
- `ASTERISK_BRIDGE_ENABLED=1`
|
||||
- `ASTERISK_IVR_FASTAGI_ENABLED=1`
|
||||
- `IVR_RUNTIME_TRUSTED_SERVICE_SUBJECTS=svc:asterisk-bridge`
|
||||
2. Make sure `ASTERISK_QUEUE_MAP_JSON` contains `voice_lab_ivr`.
|
||||
3. Provision the `prompt_audio_key` sound files in Asterisk.
|
||||
4. Call `7200` and verify:
|
||||
- DTMF completes the IVR and transfers through `mvpcc-transfer`
|
||||
- silence triggers `ivr.step.no_input`
|
||||
- retry exhaustion abandons the IVR session and falls back to the human baseline
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- `404 No active IVR flow for queue`
|
||||
- create or activate a flow for the target queue first
|
||||
- `400 Digit must be a single character 0-9`
|
||||
- send exactly one DTMF digit
|
||||
- `400 IVR session is not completed` on route preview
|
||||
- finish the session with a terminal node before previewing routing
|
||||
@@ -1,128 +0,0 @@
|
||||
# Runbook - Keycloak OIDC (Wave 2 Track 1)
|
||||
|
||||
## Scope
|
||||
|
||||
This runbook covers the enterprise identity baseline introduced in Wave 2 Track 1:
|
||||
|
||||
- Keycloak-backed OIDC login
|
||||
- signed application bearer tokens
|
||||
- controlled fallback to local login
|
||||
|
||||
It is for QA and enterprise-like environments, not the default demo flow.
|
||||
|
||||
## Modes
|
||||
|
||||
### Local / demo
|
||||
|
||||
- `OIDC_ENABLED=0`
|
||||
- `ALLOW_LEGACY_HEADER_AUTH=1`
|
||||
|
||||
Use this when:
|
||||
|
||||
- running local demos
|
||||
- using seeded local users
|
||||
- preserving existing Stage 1-4 scripts
|
||||
|
||||
### QA transition
|
||||
|
||||
- `OIDC_ENABLED=1`
|
||||
- `ALLOW_LEGACY_HEADER_AUTH=1`
|
||||
|
||||
Use this when:
|
||||
|
||||
- validating OIDC without breaking old header-based scripts
|
||||
- checking dual-mode compatibility before cutover
|
||||
|
||||
### Enterprise pilot
|
||||
|
||||
- `OIDC_ENABLED=1`
|
||||
- `ALLOW_LEGACY_HEADER_AUTH=0`
|
||||
|
||||
Use this when:
|
||||
|
||||
- OIDC is the primary login path
|
||||
- bearer-token validation is enforced end-to-end
|
||||
|
||||
## Required configuration
|
||||
|
||||
- `APP_TOKEN_SECRET`
|
||||
- `APP_TOKEN_TTL_SECONDS` (default `3600`)
|
||||
- `OIDC_ENABLED`
|
||||
- `OIDC_PROVIDER=keycloak`
|
||||
- `OIDC_ISSUER_URL`
|
||||
- `OIDC_CLIENT_ID`
|
||||
- `OIDC_CLIENT_SECRET`
|
||||
- `OIDC_REDIRECT_URI`
|
||||
- `OIDC_SCOPES` (default `openid profile email`)
|
||||
- `OIDC_ROLE_CLAIM` (default `groups`)
|
||||
- `OIDC_ROLE_MAP_JSON`
|
||||
- `ALLOW_LEGACY_HEADER_AUTH`
|
||||
|
||||
Default role map:
|
||||
|
||||
```json
|
||||
{
|
||||
"kc_admin": "admin",
|
||||
"kc_supervisor": "supervisor",
|
||||
"kc_operator": "operator",
|
||||
"kc_analyst": "analyst"
|
||||
}
|
||||
```
|
||||
|
||||
## Keycloak client settings
|
||||
|
||||
Recommended client type:
|
||||
|
||||
- confidential client
|
||||
- standard authorization code flow enabled
|
||||
- PKCE enabled
|
||||
|
||||
Redirect URI:
|
||||
|
||||
- `https://<gateway-host>/proxy/auth/auth/oidc/callback`
|
||||
|
||||
Web origins:
|
||||
|
||||
- `https://<gateway-host>`
|
||||
|
||||
## Validation flow
|
||||
|
||||
1. Apply migrations:
|
||||
- `python scripts/migrate_core_db.py`
|
||||
2. Start the stack with Track 1 env vars.
|
||||
3. Check OIDC endpoints:
|
||||
- `python scripts/oidc_smoke.py --base-url http://<gateway-host> --require-enabled`
|
||||
4. Open:
|
||||
- `http://<gateway-host>/operator`
|
||||
5. Click `Корпоративный вход`.
|
||||
6. Complete Keycloak login.
|
||||
7. Confirm the operator UI receives a session and continues using bearer auth.
|
||||
|
||||
## Failure handling
|
||||
|
||||
### OIDC health fails
|
||||
|
||||
- verify `OIDC_ISSUER_URL`
|
||||
- verify Keycloak realm is reachable from the auth-service
|
||||
- verify TLS and reverse proxy settings
|
||||
|
||||
### Login returns `403`
|
||||
|
||||
- verify user groups in Keycloak
|
||||
- verify `OIDC_ROLE_MAP_JSON`
|
||||
- verify `OIDC_ROLE_CLAIM` matches the actual claim in the ID token
|
||||
|
||||
### Existing scripts stop working
|
||||
|
||||
- in QA only, temporarily set `ALLOW_LEGACY_HEADER_AUTH=1`
|
||||
- in enterprise pilot, do not revert silently; update the script/client to bearer auth or use documented break-glass local login only
|
||||
|
||||
## Break-glass local access
|
||||
|
||||
Local `/auth/login` remains available in Track 1 for:
|
||||
|
||||
- local development
|
||||
- demo mode
|
||||
- break-glass admin access
|
||||
|
||||
It is not the primary path in enterprise mode.
|
||||
@@ -1,20 +0,0 @@
|
||||
# Load Test Plan (Stage 4)
|
||||
|
||||
## Target
|
||||
- 100 concurrent voice sessions
|
||||
- 100 concurrent digital sessions
|
||||
- Script entrypoint:
|
||||
- `python scripts/load_test.py --base-url http://localhost:8080 --voice 100 --digital 100`
|
||||
- or the profile-based equivalent:
|
||||
- `python scripts/load_test.py --base-url http://localhost:8080 --profile baseline_100_100`
|
||||
|
||||
## Metrics
|
||||
- API error rate
|
||||
- P95 response latency
|
||||
- queue processing delay
|
||||
- service availability during load
|
||||
|
||||
## Success criteria
|
||||
- No critical errors
|
||||
- Stable response times under threshold
|
||||
- No data loss in interaction lifecycle
|
||||
@@ -1,65 +0,0 @@
|
||||
# Runbook - Local Startup
|
||||
|
||||
1. Install dependencies:
|
||||
- `python -m pip install -r requirements.txt`
|
||||
2. Run tests:
|
||||
- `pytest -q`
|
||||
3. SQLite local stack:
|
||||
- `python scripts/local_stack.py start`
|
||||
4. PostgreSQL local stack:
|
||||
- start PostgreSQL and RabbitMQ locally
|
||||
- `python scripts/postgres_dev_preflight.py --env-file .env.postgres.local.template`
|
||||
- `python scripts/migrate_core_db.py`
|
||||
- `python scripts/local_stack.py start --env-file .env.postgres.local.template`
|
||||
- `python scripts/postgres_dev_preflight.py --env-file .env.postgres.local.template --base-url http://127.0.0.1:8080`
|
||||
5. Gateway URL:
|
||||
- `http://localhost:8080`
|
||||
6. Check health:
|
||||
- `GET /proxy/auth/health`
|
||||
7. Run Gate 1 quick check:
|
||||
- `python scripts/gate1_check.py`
|
||||
8. Run live Gate 1/2 smoke with auto-started services:
|
||||
- `python scripts/live_smoke_gate12.py`
|
||||
- PostgreSQL mode: `python scripts/live_smoke_gate12.py --database-url postgresql://mvp:mvp@localhost:5432/mvpcc`
|
||||
9. Run Gate 3 check:
|
||||
- Existing running stack: `python scripts/gate3_check.py`
|
||||
- Auto-start local stack: `python scripts/gate3_check.py --auto-start`
|
||||
10. Run Gate 4 automated checks:
|
||||
- `python scripts/gate4_check.py`
|
||||
- Keep artifacts: `python scripts/gate4_check.py --keep-artifacts`
|
||||
11. Run UAT preflight:
|
||||
- Existing running stack: `python scripts/uat_preflight.py --base-url http://localhost:8080`
|
||||
- Auto-start local stack: `python scripts/uat_preflight.py --auto-start`
|
||||
12. Run UAT dry-run package:
|
||||
- Existing running stack: `python scripts/uat_dry_run.py --base-url http://localhost:8080 --update-defect-register`
|
||||
- Auto-start local stack: `python scripts/uat_dry_run.py --auto-start --update-defect-register`
|
||||
13. Optional OIDC smoke-check:
|
||||
- `python scripts/oidc_smoke.py --base-url http://localhost:8080`
|
||||
- Require enterprise path enabled: `python scripts/oidc_smoke.py --base-url http://localhost:8080 --require-enabled`
|
||||
14. Open operator UI:
|
||||
- `http://localhost:8080/operator`
|
||||
15. Optional dedicated Wave 2 shells:
|
||||
- `http://localhost:8080/supervisor`
|
||||
- `http://localhost:8080/admin`
|
||||
16. Reset local runtime artifacts when needed:
|
||||
- `powershell -ExecutionPolicy Bypass -File scripts\clean_workspace.ps1`
|
||||
|
||||
## Runtime artifacts
|
||||
|
||||
- `.data*`, `.local_stack`, and `.artifacts` are regeneratable local runtime directories.
|
||||
- PostgreSQL local profile keeps runtime artifacts in `.data_local_pg` by default.
|
||||
- Use `scripts/clean_workspace.ps1` to clear local DBs, logs, cached recordings, and test/runtime leftovers without touching source files or backups.
|
||||
|
||||
## Local auth modes
|
||||
|
||||
- Demo/dev default:
|
||||
- `OIDC_ENABLED=0`
|
||||
- `ALLOW_LEGACY_HEADER_AUTH=1`
|
||||
- Enterprise-like local validation:
|
||||
- set `OIDC_ENABLED=1`
|
||||
- set `ALLOW_LEGACY_HEADER_AUTH=0`
|
||||
- configure `APP_TOKEN_SECRET`, `OIDC_ISSUER_URL`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`, `OIDC_REDIRECT_URI`
|
||||
|
||||
See also:
|
||||
|
||||
- [keycloak-oidc.md](/e:/Zhan/docs/runbooks/keycloak-oidc.md)
|
||||
@@ -0,0 +1,94 @@
|
||||
# Руководство по локальному запуску (Local Setup)
|
||||
|
||||
В этом документе описаны шаги для поднятия проекта на рабочей машине разработчика или для проведения презентации (демо). Проект поддерживает как легковесный запуск на SQLite, так и масштабируемый запуск с использованием PostgreSQL и RabbitMQ.
|
||||
|
||||
## 1. Системные требования
|
||||
- **Python 3.10+** (для микросервисов)
|
||||
- **Docker и Docker Compose** (для запуска PostgreSQL, RabbitMQ и т.д.)
|
||||
- **PowerShell** (основные launch-скрипты написаны на нём)
|
||||
|
||||
## 2. Быстрый старт: Режим ДЕМО (Автоматически)
|
||||
|
||||
Самый удобный способ поднять весь стек контакт-центра одной командой. Скрипт запустит сервисы, прогонит смоук-тесты и заполнит базу моковыми данными:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\prepare_demo.ps1
|
||||
```
|
||||
|
||||
**Что делает `prepare_demo.ps1`**:
|
||||
- Поднимает все микросервисы проекта в фоне (логи складываются в папку `.local_stack\logs`).
|
||||
- Создает тестового клиента и историю его взаимодействия.
|
||||
- Симулирует звонок с IVR и создает запись в `recording-service`.
|
||||
- Наливает статьи Базы Знаний.
|
||||
- Забрасывает метрики для KPI отчетов в `reporting-service`.
|
||||
|
||||
**Остановка и очистка:**
|
||||
```powershell
|
||||
# Остановить все фоновые процессы сервисов
|
||||
powershell -ExecutionPolicy Bypass -File scripts\stop_all_local.ps1
|
||||
|
||||
# Полностью очистить рабочее пространство сервера (удалить временные БД и артефакты)
|
||||
powershell -ExecutionPolicy Bypass -File scripts\clean_workspace.ps1
|
||||
```
|
||||
|
||||
## 3. Ручной запуск (Local Development)
|
||||
|
||||
Если вы разрабатываете конкретный микросервис и вам нужен горячий рестарт (Hot Reload), поднимайте сервисы вручную.
|
||||
|
||||
### Шаг 3.1: Установка зависимостей
|
||||
```bash
|
||||
python -m pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### Шаг 3.2: Запуск инфраструктуры (Docker)
|
||||
Если вы не хотите использовать fallback-режим с SQLite, поднимите PostgreSQL и системные сервисы:
|
||||
```bash
|
||||
cd deployment
|
||||
docker compose up -d postgres
|
||||
cd ..
|
||||
```
|
||||
|
||||
### Шаг 3.3: Миграции Базы Данных
|
||||
Перед запуском бизнес-логики необходимо проинициализировать схему БД:
|
||||
```bash
|
||||
python scripts/migrate_core_db.py
|
||||
```
|
||||
|
||||
### Шаг 3.4: Запуск Gateway и Микросервисов
|
||||
Для маршрутизации запросов обязательно должен работать `api-gateway`:
|
||||
```bash
|
||||
uvicorn gateway.app:app --reload --port 8080
|
||||
```
|
||||
Далее в новых консолях вы можете точечно поднимать необходимые вам микросервисы (по умолчанию Gateway проксирует запросы на порты `8001`, `8004` и т.д.):
|
||||
```bash
|
||||
uvicorn services.auth_service.app:app --reload --port 8001
|
||||
uvicorn services.interaction_service.app:app --reload --port 8004
|
||||
```
|
||||
|
||||
## 4. Точки входа (Пользовательские интерфейсы)
|
||||
|
||||
Если `gateway.app` успешно запущен на порту 8080, вы можете попасть в веб-интерфейсы платформы по следующим адресам:
|
||||
|
||||
- **Рабочее место Оператора**: [http://localhost:8080/operator](http://localhost:8080/operator)
|
||||
- **Панель Супервизора**: [http://localhost:8080/supervisor](http://localhost:8080/supervisor)
|
||||
- **Панель Администратора**: [http://localhost:8080/admin](http://localhost:8080/admin)
|
||||
- **Аналитика (Дашборды)**: [http://localhost:8080/analyst](http://localhost:8080/analyst)
|
||||
|
||||
## 5. Полезные скрипты
|
||||
|
||||
В папке `scripts/` лежит множество полезных инструментов для разработки:
|
||||
|
||||
```bash
|
||||
# Прогон всех автотестов
|
||||
pytest -q
|
||||
|
||||
# Смоук-чек авторизации (проверка выдачи JWT токенов)
|
||||
python scripts/oidc_smoke.py --base-url http://localhost:8080
|
||||
|
||||
# Снятие и распаковка бэкапа (дамп данных лок. среды)
|
||||
powershell -ExecutionPolicy Bypass -File scripts\backup_data.ps1
|
||||
powershell -ExecutionPolicy Bypass -File scripts\restore_data.ps1 -BackupZip e:\Zhan\backups\mvp_cc_data_YYMMDD.zip
|
||||
|
||||
# Нагрузочное тестирование (позволяет сгенерировать фейковый трафик звонков)
|
||||
python scripts/load_test.py --base-url http://localhost:8080 --profile baseline_100_100
|
||||
```
|
||||
@@ -1,39 +0,0 @@
|
||||
# Management Demo Brief
|
||||
|
||||
Use this note during the meeting so the demo stays short and clear.
|
||||
|
||||
## One-minute version
|
||||
|
||||
1. Open `http://localhost:8080/operator`.
|
||||
2. Log in as `admin / admin123`.
|
||||
3. Show one closed interaction and one active interaction.
|
||||
4. Show the supervisor summary.
|
||||
5. Show the KPI summary.
|
||||
|
||||
## Three-minute version
|
||||
|
||||
1. Show the demo checklist at the top of the screen.
|
||||
2. Explain that the stand is started by one command and loaded with demo data.
|
||||
3. Open customers and interactions.
|
||||
4. Show assignment, transfer to second line, and closing.
|
||||
5. Trigger the voice, Telegram, Webchat, and Email channel checks.
|
||||
6. Search KB for `demo-showcase`.
|
||||
7. Show supervisor and KPI summary cards.
|
||||
|
||||
## What to say by block
|
||||
|
||||
- Session: role-based access and a working login flow.
|
||||
- Customers: the platform stores and finds client cards.
|
||||
- Interactions: the request can be created, assigned, escalated, and closed.
|
||||
- Integrations: the system accepts voice, Telegram, Webchat, and Email events.
|
||||
- Knowledge Base: the operator can quickly find a prepared answer.
|
||||
- Supervisor: the lead sees current queue and agent state.
|
||||
- KPI: the system already calculates core operational metrics.
|
||||
|
||||
## Five short answers
|
||||
|
||||
- What is ready: MVP with operator flow, integrations, KB, supervisor, KPI, and the first Wave 2 webchat/email channels.
|
||||
- What is not yet final: this is a pilot-ready MVP, not a full production rollout.
|
||||
- What is the stack: Python/FastAPI backend, browser UI, microservice layout.
|
||||
- What about telephony: the voice integration contour is ready and demonstrated by event intake.
|
||||
- What is next: a separate Wave 2 with SSO, extra channels, expanded admin UX, and scaling.
|
||||
@@ -1,88 +0,0 @@
|
||||
# Runbook - Pilot UAT
|
||||
|
||||
## Participants
|
||||
|
||||
- Operators
|
||||
- Supervisor
|
||||
- Analyst
|
||||
- Admin
|
||||
- Business owner
|
||||
- IT owner
|
||||
|
||||
## Scope Rules
|
||||
|
||||
- Freeze the pilot scope using `docs/gates/mvp-pilot-baseline.md`.
|
||||
- Do not record out-of-scope feature requests as MVP defects.
|
||||
- Route deferred requests to `docs/roadmap/05-wave2-backlog.md`.
|
||||
|
||||
## Preparation
|
||||
|
||||
1. Run environment preflight:
|
||||
- Existing environment: `python scripts/uat_preflight.py --base-url http://<gateway-host>:8080`
|
||||
- Local auto-start: `python scripts/uat_preflight.py --auto-start`
|
||||
2. Run UAT dry-run package generator:
|
||||
- Existing environment: `python scripts/uat_dry_run.py --base-url http://<gateway-host>:8080 --update-defect-register`
|
||||
- Local auto-start: `python scripts/uat_dry_run.py --auto-start --update-defect-register`
|
||||
3. Create session protocol from template or generated artifact:
|
||||
- `docs/uat/session-template.md`
|
||||
- Recommended bundle generator:
|
||||
- `python scripts/uat_manual_prepare.py --environment-url http://<pilot-gateway>:8080`
|
||||
4. Prepare defect log file:
|
||||
- `docs/uat/defect-log-template.csv`
|
||||
5. Confirm the pilot gateway URL and the build under test in the session protocol.
|
||||
|
||||
## Mandatory scenarios
|
||||
|
||||
- Execute all scenarios from:
|
||||
- `docs/uat/scenario-checklist.md`
|
||||
- Required scope:
|
||||
- auth login and RBAC deny
|
||||
- customer create/search
|
||||
- voice lifecycle
|
||||
- assignment and escalation
|
||||
- routing and timeline verification
|
||||
- voice event intake
|
||||
- Telegram lifecycle
|
||||
- supervisor realtime
|
||||
- KPI report and CSV
|
||||
- KB usage in handling flow
|
||||
- backup/restore verification
|
||||
- load test baseline (`100 voice + 100 digital`)
|
||||
|
||||
## Defect triage
|
||||
|
||||
- `P1`: acceptance impossible, fix in MVP immediately
|
||||
- `P2`: key journey broken, fix in MVP immediately
|
||||
- `P3`: acceptable with limitation, defer unless it blocks sign-off
|
||||
- `P4`: cosmetic or backlog-only
|
||||
|
||||
Only `P1` and `P2` defects belong to the MVP remediation cycle.
|
||||
|
||||
## Evidence package
|
||||
|
||||
- Preflight report from `docs/uat/evidence/`
|
||||
- Completed scenario checklist
|
||||
- Completed session protocol
|
||||
- Filled defect log
|
||||
- Signed sheet from `docs/uat/signoff-template.md`
|
||||
- Updated `docs/gates/p1-p2-defects.md`
|
||||
|
||||
## Formal closure
|
||||
|
||||
After manual signatures are collected and no open `P1/P2` items remain:
|
||||
|
||||
- Optional validation only:
|
||||
- `python scripts/finalize_mvp_pilot.py --session-dir docs/uat/evidence/manual_<session-id> --dry-run`
|
||||
- Run:
|
||||
- `python scripts/finalize_mvp_pilot.py --session-dir docs/uat/evidence/manual_<session-id>`
|
||||
- This updates:
|
||||
- `docs/gates/gate-04-pilot-hardening.md`
|
||||
- `docs/gates/p1-p2-defects.md`
|
||||
- `docs/releases/v1.0.0-mvp-accepted.md`
|
||||
|
||||
## Exit criteria
|
||||
|
||||
- No open P1 defects
|
||||
- No open P2 defects
|
||||
- Critical user journeys approved
|
||||
- Sign-off captured from business and IT owners
|
||||
@@ -1,275 +0,0 @@
|
||||
# PostgreSQL Dev Cutover
|
||||
|
||||
Этот runbook описывает безопасный перевод `dev`-окружения с SQLite на PostgreSQL без переноса старых данных.
|
||||
|
||||
## Что считается "без простоя"
|
||||
|
||||
В этом проекте без переноса SQLite-данных безостановочный cutover означает не hot-swap одной и той же базы, а:
|
||||
|
||||
1. поднять новый PostgreSQL-backed stack параллельно;
|
||||
2. прогнать smoke на новом stack;
|
||||
3. переключить dev-трафик на новый gateway;
|
||||
4. оставить старый SQLite stack живым до подтверждения GO;
|
||||
5. при проблеме вернуть трафик обратно без восстановления данных.
|
||||
|
||||
Если сейчас у dev только один instance и нет возможности держать parallel stack, нужен короткий maintenance window.
|
||||
|
||||
## Scope
|
||||
|
||||
Включает:
|
||||
|
||||
- подготовку PostgreSQL и RabbitMQ;
|
||||
- применение SQL migrations;
|
||||
- запуск сервисов в `SCHEMA_MANAGEMENT_MODE=migrations`;
|
||||
- smoke-проверки до и после переключения;
|
||||
- rollback на старый SQLite-backed dev stack.
|
||||
|
||||
Не включает:
|
||||
|
||||
- перенос данных из SQLite;
|
||||
- cleanup legacy SQLite-path;
|
||||
- production cutover.
|
||||
|
||||
## Required Inputs
|
||||
|
||||
- `DEV_GATEWAY_OLD`
|
||||
- `DEV_GATEWAY_NEW`
|
||||
- `POSTGRES_DATABASE_URL`
|
||||
- `APP_TOKEN_SECRET`
|
||||
- `PUBLIC_SWITCH_METHOD`
|
||||
- DNS
|
||||
- reverse proxy
|
||||
- load balancer route
|
||||
- `RABBITMQ_URL`
|
||||
|
||||
Рекомендуемые значения для локально-управляемого dev:
|
||||
|
||||
- `POSTGRES_DATABASE_URL=postgresql://mvp:mvp@<postgres-host>:5432/mvpcc`
|
||||
- `SCHEMA_MANAGEMENT_MODE=migrations`
|
||||
- env template: [\.env.postgres.local.template](/e:/Zhan/.env.postgres.local.template)
|
||||
|
||||
## Phase 0 - Preconditions
|
||||
|
||||
Подтвердить перед началом:
|
||||
|
||||
1. Текущий SQLite-backed dev stack стабилен.
|
||||
2. Перенос данных не нужен.
|
||||
3. PostgreSQL доступен по сети с host, где запускаются сервисы.
|
||||
4. RabbitMQ доступен по сети, если в dev нужен `EVENT_BUS_ENABLED=1`.
|
||||
5. Новая dev-конфигурация использует:
|
||||
- `DATABASE_URL=postgresql://...`
|
||||
- `SCHEMA_MANAGEMENT_MODE=migrations`
|
||||
6. Старый SQLite stack не останавливается до завершения smoke на новом stack.
|
||||
|
||||
## Phase 1 - Preflight
|
||||
|
||||
На будущем PostgreSQL-backed dev host:
|
||||
|
||||
1. Проверить Python зависимости:
|
||||
|
||||
```powershell
|
||||
python -m pip install -r requirements.txt
|
||||
```
|
||||
|
||||
2. Подготовить env-файл:
|
||||
- скопировать [\.env.postgres.local.template](/e:/Zhan/.env.postgres.local.template)
|
||||
- заполнить `DATABASE_URL`, `APP_TOKEN_SECRET`, интеграционные URL и при необходимости AI/Telegram/WhatsApp переменные
|
||||
|
||||
3. Запустить DB-only preflight:
|
||||
|
||||
```powershell
|
||||
python scripts\postgres_dev_preflight.py --env-file .env.postgres.local.template
|
||||
```
|
||||
|
||||
4. Убедиться, что mode не legacy:
|
||||
|
||||
```text
|
||||
SCHEMA_MANAGEMENT_MODE=migrations
|
||||
```
|
||||
|
||||
5. Применить миграции в PostgreSQL:
|
||||
|
||||
```powershell
|
||||
python scripts\migrate_core_db.py
|
||||
```
|
||||
|
||||
Ожидается:
|
||||
|
||||
- миграции применились без ошибок;
|
||||
- в PostgreSQL появилась таблица `schema_migrations`;
|
||||
- сервисы ещё не стартовали.
|
||||
|
||||
## Phase 2 - Bring Up Parallel Stack
|
||||
|
||||
Поднять новый stack параллельно старому.
|
||||
|
||||
Если запуск локальный:
|
||||
|
||||
```powershell
|
||||
python scripts\local_stack.py start `
|
||||
--runtime-dir .local_stack_pg `
|
||||
--env-file .env.postgres.local.template `
|
||||
--force-restart
|
||||
```
|
||||
|
||||
Если это dev server / VM:
|
||||
|
||||
1. развернуть те же сервисы в отдельный deployment set;
|
||||
2. прокинуть им:
|
||||
- `DATABASE_URL=postgresql://...`
|
||||
- `SCHEMA_MANAGEMENT_MODE=migrations`
|
||||
3. не направлять внешний dev-трафик на новый gateway до завершения smoke.
|
||||
|
||||
Важно:
|
||||
|
||||
- новый stack должен использовать отдельный runtime/log directory;
|
||||
- старый SQLite-backed stack остаётся доступным;
|
||||
- нельзя смешивать новый dev-трафик со старым gateway до smoke.
|
||||
|
||||
## Phase 3 - Smoke Before Switch
|
||||
|
||||
Выполнить проверки на `DEV_GATEWAY_NEW`.
|
||||
|
||||
Сначала прогнать HTTP-aware preflight:
|
||||
|
||||
```powershell
|
||||
python scripts\postgres_dev_preflight.py `
|
||||
--env-file .env.postgres.local.template `
|
||||
--base-url http://127.0.0.1:8080
|
||||
```
|
||||
|
||||
1. Health:
|
||||
|
||||
```powershell
|
||||
python scripts\live_smoke_gate12.py --base-url http://<DEV_GATEWAY_NEW> --database-url <POSTGRES_DATABASE_URL>
|
||||
```
|
||||
|
||||
2. Focused PostgreSQL smoke:
|
||||
|
||||
```powershell
|
||||
pytest -q tests/test_postgres_readiness.py
|
||||
```
|
||||
|
||||
3. Дополнительно проверить вручную:
|
||||
|
||||
- `GET /proxy/auth/health`
|
||||
- `GET /proxy/interaction/health`
|
||||
- `GET /proxy/voice/health`
|
||||
- `GET /proxy/ai-voice-runtime/health`
|
||||
- открыть `http://<DEV_GATEWAY_NEW>/operator`
|
||||
|
||||
4. Если в dev включён event bus:
|
||||
|
||||
- проверить подключение к RabbitMQ;
|
||||
- убедиться, что `event-bus-service` поднимается в `healthy`;
|
||||
- убедиться, что новые записи появляются в `event_outbox`.
|
||||
|
||||
GO в следующую фазу только если:
|
||||
|
||||
- health green;
|
||||
- smoke green;
|
||||
- новый stack не пытается auto-create schema на старте;
|
||||
- нет ошибок вида `run python scripts/migrate_core_db.py` после фактического применения миграций.
|
||||
|
||||
## Phase 4 - Traffic Switch
|
||||
|
||||
Переключение выполняется только после успешного Phase 3.
|
||||
|
||||
Рекомендуемый порядок:
|
||||
|
||||
1. уменьшить TTL у dev DNS / подготовить proxy route заранее;
|
||||
2. переключить `DEV_GATEWAY` на `DEV_GATEWAY_NEW`;
|
||||
3. не останавливать старый SQLite stack;
|
||||
4. сразу после switch выполнить быстрый post-switch smoke.
|
||||
|
||||
Примеры post-switch smoke:
|
||||
|
||||
```powershell
|
||||
python scripts\live_smoke_gate12.py --base-url http://<DEV_GATEWAY_PUBLIC> --database-url <POSTGRES_DATABASE_URL>
|
||||
```
|
||||
|
||||
Проверить руками:
|
||||
|
||||
- логин;
|
||||
- создание interaction;
|
||||
- digital thread/message flow;
|
||||
- voice event ingest;
|
||||
- создание voice AI session.
|
||||
|
||||
## Phase 5 - Stabilization Window
|
||||
|
||||
В течение первых 15-30 минут после switch:
|
||||
|
||||
1. наблюдать логи gateway и сервисов;
|
||||
2. следить за ошибками подключения к PostgreSQL;
|
||||
3. следить за ошибками `schema mismatch`;
|
||||
4. следить за ошибками блокировок/unique conflict в voice transcript path;
|
||||
5. сравнивать user-visible поведение со старым dev stack.
|
||||
|
||||
Если всё стабильно:
|
||||
|
||||
1. объявить GO;
|
||||
2. зафиксировать, что `dev` теперь PostgreSQL-backed;
|
||||
3. оставить SQLite stack выключенным, но не удалённым до конца рабочего дня.
|
||||
|
||||
## Rollback
|
||||
|
||||
Rollback делается только через возврат трафика на старый SQLite-backed stack.
|
||||
|
||||
Триггеры rollback:
|
||||
|
||||
- новый gateway не проходит smoke;
|
||||
- сервисы падают на startup;
|
||||
- критичный API path broken;
|
||||
- UI не работает;
|
||||
- ошибки подключения к PostgreSQL не устраняются быстро;
|
||||
- event bus / voice path деградирует.
|
||||
|
||||
Шаги rollback:
|
||||
|
||||
1. вернуть `DEV_GATEWAY_PUBLIC` на `DEV_GATEWAY_OLD`;
|
||||
2. убедиться, что старый SQLite stack всё ещё жив;
|
||||
3. выполнить быстрый smoke на старом stack;
|
||||
4. запретить новый трафик на PostgreSQL-backed stack;
|
||||
5. собрать логи с нового stack.
|
||||
|
||||
Rollback успешен, если:
|
||||
|
||||
- dev UI снова открывается на старом gateway;
|
||||
- CRUD и smoke снова зелёные;
|
||||
- команда работает на старом dev без дополнительных действий.
|
||||
|
||||
## Cutover Sheet
|
||||
|
||||
Перед cutover заполнить:
|
||||
|
||||
- время старта;
|
||||
- кто выполняет switch;
|
||||
- `DEV_GATEWAY_OLD`;
|
||||
- `DEV_GATEWAY_NEW`;
|
||||
- `POSTGRES_DATABASE_URL`;
|
||||
- результат preflight;
|
||||
- результат smoke до switch;
|
||||
- время switch;
|
||||
- результат smoke после switch;
|
||||
- GO / ROLLBACK;
|
||||
- короткий список замечаний.
|
||||
|
||||
## Minimal Command Sequence
|
||||
|
||||
Если нужен самый короткий practical path для dev:
|
||||
|
||||
```powershell
|
||||
python scripts\migrate_core_db.py
|
||||
python scripts\postgres_dev_preflight.py --env-file .env.postgres.local.template
|
||||
python scripts\local_stack.py start --runtime-dir .local_stack_pg --env-file .env.postgres.local.template --force-restart
|
||||
python scripts\postgres_dev_preflight.py --env-file .env.postgres.local.template --base-url http://127.0.0.1:8080
|
||||
pytest -q tests/test_postgres_readiness.py
|
||||
python scripts\live_smoke_gate12.py --database-url postgresql://mvp:mvp@localhost:5432/mvpcc
|
||||
```
|
||||
|
||||
После этого:
|
||||
|
||||
1. переключить dev gateway/public route на новый stack;
|
||||
2. прогнать smoke ещё раз уже через публичный dev URL;
|
||||
3. оставить старый SQLite stack как rollback target.
|
||||
@@ -1,88 +0,0 @@
|
||||
# Runbook - Shared Docker Host PostgreSQL for call-center
|
||||
|
||||
Use this runbook when the server already hosts other Docker projects and `call-center` still runs on SQLite. This flow creates an isolated PostgreSQL instance for `call-center` only and does not restart the current application stack.
|
||||
|
||||
## Assets
|
||||
|
||||
- Compose file: `deployment/docker-compose.postgres.server.yml`
|
||||
- Secret env file: `/home/mvpcc/call-center/.env.postgres.server`
|
||||
- Data directory: `/home/mvpcc/call-center/.data_pg`
|
||||
- Host-local PostgreSQL port: `127.0.0.1:5434`
|
||||
|
||||
Future application connection string:
|
||||
|
||||
- containers on the same Docker host: `postgresql://mvpcc_app:<secret>@call-center-postgres:5432/mvpcc`
|
||||
- host-local checks: `postgresql://mvpcc_app:<secret>@127.0.0.1:5434/mvpcc`
|
||||
|
||||
## Bring-up
|
||||
|
||||
From the repository root on the target server:
|
||||
|
||||
```bash
|
||||
cd /home/mvpcc/call-center
|
||||
python3 - <<'PY'
|
||||
from pathlib import Path
|
||||
import secrets
|
||||
|
||||
target = Path(".env.postgres.server")
|
||||
target.write_text(
|
||||
"POSTGRES_DB=mvpcc\n"
|
||||
"POSTGRES_USER=mvpcc_app\n"
|
||||
f"POSTGRES_PASSWORD={secrets.token_urlsafe(32)}\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
PY
|
||||
chmod 600 .env.postgres.server
|
||||
mkdir -p .data_pg
|
||||
chmod 700 .data_pg
|
||||
docker compose -f deployment/docker-compose.postgres.server.yml up -d
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
Container status and logs:
|
||||
|
||||
```bash
|
||||
cd /home/mvpcc/call-center
|
||||
docker compose -f deployment/docker-compose.postgres.server.yml ps
|
||||
docker compose -f deployment/docker-compose.postgres.server.yml logs postgres --tail 50
|
||||
docker exec call-center-postgres pg_isready -U mvpcc_app -d mvpcc
|
||||
```
|
||||
|
||||
Host-local connection check without installing `psql` on the server:
|
||||
|
||||
```bash
|
||||
cd /home/mvpcc/call-center
|
||||
set -a
|
||||
. ./.env.postgres.server
|
||||
set +a
|
||||
docker run --rm --network host \
|
||||
-e PGPASSWORD="$POSTGRES_PASSWORD" \
|
||||
postgres:16-alpine \
|
||||
psql -h 127.0.0.1 -p 5434 -U "$POSTGRES_USER" -d "$POSTGRES_DB" \
|
||||
-Atqc "select current_database(), current_user;"
|
||||
```
|
||||
|
||||
Port exposure check:
|
||||
|
||||
```bash
|
||||
ss -ltn | grep 5434
|
||||
```
|
||||
|
||||
Expected result:
|
||||
|
||||
- the container is `healthy`
|
||||
- `pg_isready` returns `accepting connections`
|
||||
- the SQL check returns `mvpcc|mvpcc_app`
|
||||
- `ss` shows `127.0.0.1:5434`, not `0.0.0.0:5434`
|
||||
|
||||
## Scope guardrails
|
||||
|
||||
Do not do these actions in this step:
|
||||
|
||||
- do not edit `deployment/docker-compose.server.yml`
|
||||
- do not switch `DATABASE_URL` for the running `call-center` services
|
||||
- do not run application migrations against the new PostgreSQL instance yet
|
||||
- do not touch existing PostgreSQL containers used by other projects
|
||||
|
||||
When a second Dockerized `call-center` stack is added later on the same host, connect it to the external network `call-center-postgres_default` and use `call-center-postgres:5432` from the application containers.
|
||||
@@ -1,105 +0,0 @@
|
||||
# Runbook - Parallel PostgreSQL-backed Stack on a Shared Docker Host
|
||||
|
||||
Use this runbook after the isolated PostgreSQL server from `postgres-server-docker.md` is already running. The goal is to bring up a second `call-center` stack on PostgreSQL without stopping the current SQLite-backed stack on port `8080`.
|
||||
|
||||
## Assets
|
||||
|
||||
- Compose file: `deployment/docker-compose.parallel.server.yml`
|
||||
- Base env: `/home/mvpcc/call-center-pg/.env.production`
|
||||
- Override env: `/home/mvpcc/call-center-pg/.env.postgres.parallel.server`
|
||||
- Runtime data dir: `/home/mvpcc/call-center-pg/.data_pg_parallel`
|
||||
- Gateway port: `18080`
|
||||
|
||||
This parallel stack intentionally disables active external integrations so it stays passive while the SQLite-backed stack remains live:
|
||||
|
||||
- Telegram bot disabled
|
||||
- WhatsApp outbound disabled
|
||||
- Asterisk bridge disabled
|
||||
- FastAGI disabled
|
||||
- AI voice disabled
|
||||
|
||||
The parallel stack reaches PostgreSQL over the external Docker network created by `deployment/docker-compose.postgres.server.yml`, using the hostname `call-center-postgres` on port `5432`.
|
||||
|
||||
## Server bring-up
|
||||
|
||||
Prepare a separate build workspace from the current repository checkout:
|
||||
|
||||
```bash
|
||||
rm -rf /home/mvpcc/call-center-pg
|
||||
mkdir -p /home/mvpcc/call-center-pg
|
||||
```
|
||||
|
||||
Copy the current `.env.production` from the live SQLite stack, then add a PostgreSQL override file:
|
||||
|
||||
```bash
|
||||
cd /home/mvpcc/call-center-pg
|
||||
cp /home/mvpcc/call-center/.env.production .env.production
|
||||
cp .env.postgres.parallel.server.template .env.postgres.parallel.server
|
||||
```
|
||||
|
||||
Update `.env.postgres.parallel.server`:
|
||||
|
||||
- replace `<server-host>` with the real server host or IP
|
||||
- replace the PostgreSQL password placeholder with the password from `/home/mvpcc/call-center/.env.postgres.server`
|
||||
- keep `DATABASE_URL` pointed at `call-center-postgres:5432`, not `127.0.0.1:5434`
|
||||
|
||||
Build the image and prepare the runtime directory:
|
||||
|
||||
```bash
|
||||
cd /home/mvpcc/call-center-pg
|
||||
mkdir -p .data_pg_parallel
|
||||
docker compose -f deployment/docker-compose.parallel.server.yml build
|
||||
```
|
||||
|
||||
The parallel compose builds and uses the dedicated image tag `call-center-app:pg-parallel`.
|
||||
|
||||
Run migrations before starting the stack:
|
||||
|
||||
```bash
|
||||
cd /home/mvpcc/call-center-pg
|
||||
docker compose -f deployment/docker-compose.parallel.server.yml run --rm auth-service python scripts/migrate_core_db.py
|
||||
```
|
||||
|
||||
Bring up the PostgreSQL-backed stack:
|
||||
|
||||
```bash
|
||||
cd /home/mvpcc/call-center-pg
|
||||
docker compose -f deployment/docker-compose.parallel.server.yml up -d
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
Health and status:
|
||||
|
||||
```bash
|
||||
cd /home/mvpcc/call-center-pg
|
||||
docker compose -f deployment/docker-compose.parallel.server.yml ps
|
||||
curl -fsS http://127.0.0.1:18080/health
|
||||
curl -fsS http://127.0.0.1:18080/proxy/auth/health
|
||||
curl -fsS http://127.0.0.1:8080/health
|
||||
```
|
||||
|
||||
Focused PostgreSQL preflight from inside the parallel stack:
|
||||
|
||||
```bash
|
||||
cd /home/mvpcc/call-center-pg
|
||||
docker compose -f deployment/docker-compose.parallel.server.yml run --rm auth-service \
|
||||
sh -lc 'python scripts/postgres_dev_preflight.py --database-url "$DATABASE_URL" --base-url http://api-gateway:8000'
|
||||
```
|
||||
|
||||
Expected result:
|
||||
|
||||
- the new stack is reachable at `http://<server-host>:18080`
|
||||
- the old SQLite-backed stack stays reachable at `http://<server-host>:8080`
|
||||
- no host ports except `18080` are added for the parallel stack
|
||||
- the PostgreSQL-backed containers stay healthy
|
||||
- application containers talk to PostgreSQL over Docker network `call-center-postgres_default`
|
||||
|
||||
## Scope guardrails
|
||||
|
||||
Do not do these actions in this step:
|
||||
|
||||
- do not stop or restart the existing `call-center` compose project
|
||||
- do not change `/home/mvpcc/call-center/deployment/docker-compose.server.yml`
|
||||
- do not reuse the existing `call-center-app:local` image tag for the new stack
|
||||
- do not re-enable Telegram, WhatsApp, or Asterisk on the parallel stack until cutover is planned
|
||||
@@ -1,65 +0,0 @@
|
||||
# Local Recordings Runbook
|
||||
|
||||
Use this flow to validate Track 4 locally.
|
||||
|
||||
## Storage location
|
||||
|
||||
- Default root: `.data_local\recordings`
|
||||
- Override with `CC_RECORDINGS_DIR`
|
||||
|
||||
Files are copied into a managed layout:
|
||||
|
||||
- `YYYY\MM\DD\rec_<id>_<file-name>`
|
||||
|
||||
## Import flow
|
||||
|
||||
1. Submit a `recording.ready` voice event with:
|
||||
- `payload.source_path`
|
||||
- optional `file_name`, `mime_type`, `duration_seconds`, `recorded_at`
|
||||
2. Import it through:
|
||||
|
||||
```text
|
||||
POST /proxy/recording/recordings/import-from-voice-event/{event_id}
|
||||
```
|
||||
|
||||
You can also register directly by server path:
|
||||
|
||||
```text
|
||||
POST /proxy/recording/recordings/register
|
||||
```
|
||||
|
||||
## Review in supervisor shell
|
||||
|
||||
1. Open `http://localhost:8080/supervisor`
|
||||
2. Go to `Recordings`
|
||||
3. Load recordings or import by `voice_event_id`
|
||||
4. Select a row to open metadata and preview
|
||||
5. Use:
|
||||
- `Open` for metadata + playback
|
||||
- `Download` for file export
|
||||
- `Archive` to mark the recording as archived
|
||||
|
||||
## Missing file handling
|
||||
|
||||
If the DB row exists but the stored file is gone:
|
||||
|
||||
- content request returns `410 Gone`
|
||||
- the row status becomes `missing`
|
||||
- the metadata remains available for diagnosis
|
||||
|
||||
## Limits
|
||||
|
||||
- Default max size: `25 MB`
|
||||
- Override with `RECORDING_MAX_BYTES`
|
||||
|
||||
## Demo seed
|
||||
|
||||
`scripts\prepare_demo.ps1` now creates:
|
||||
|
||||
- one sample `.wav` file
|
||||
- one `recording.ready` voice event
|
||||
- one imported managed recording
|
||||
|
||||
The generated `recording_id` is stored in:
|
||||
|
||||
- `.local_stack\demo-seed-summary.json`
|
||||
@@ -1,77 +0,0 @@
|
||||
# Runbook - Track 10 Live Voice Reliability
|
||||
|
||||
Use this runbook to validate lifecycle latency after Track 9 integration.
|
||||
|
||||
## Preconditions
|
||||
|
||||
- Platform stack is running.
|
||||
- Asterisk bridge is connected (`/proxy/asterisk-bridge/asterisk/status`).
|
||||
- Real lab calls are already made (`1001 -> 7000`).
|
||||
|
||||
## 1) Generate latency report
|
||||
|
||||
```powershell
|
||||
python scripts\track10_voice_latency_report.py `
|
||||
--database-url sqlite:///.data_local/mvp_cc.db `
|
||||
--since-hours 8 `
|
||||
--breach-mode direct `
|
||||
--json-out .artifacts/track10/voice-latency-latest.json
|
||||
```
|
||||
|
||||
For post-fix short-window validation use minutes:
|
||||
|
||||
```powershell
|
||||
python scripts\track10_voice_latency_report.py `
|
||||
--database-url sqlite:///.data_local/mvp_cc.db `
|
||||
--since-minutes 30 `
|
||||
--breach-mode direct `
|
||||
--json-out .artifacts/track10/voice-latency-30m.json
|
||||
```
|
||||
|
||||
Optional strict gate mode:
|
||||
|
||||
```powershell
|
||||
python scripts\track10_voice_latency_report.py `
|
||||
--database-url sqlite:///.data_local/mvp_cc.db `
|
||||
--since-hours 8 `
|
||||
--max-started-to-ended-seconds 30 `
|
||||
--max-ended-to-recording-seconds 45 `
|
||||
--breach-mode direct `
|
||||
--fail-on-breach
|
||||
```
|
||||
|
||||
## 2) Review report fields
|
||||
|
||||
Mandatory fields:
|
||||
- `summary.total_calls`
|
||||
- `summary.active_no_end`
|
||||
- `summary.ended_no_recording_upload`
|
||||
- `summary.start_to_end.p95`
|
||||
- `summary.end_to_recording_upload.p95`
|
||||
- `summary.ended_direct_calls`
|
||||
- `summary.ended_reconciled_calls`
|
||||
- `summary.direct_start_to_end.p95`
|
||||
- `summary.direct_end_to_recording_upload.p95`
|
||||
|
||||
## 3) Save acceptance evidence
|
||||
|
||||
Create a timestamp folder:
|
||||
|
||||
`docs/acceptance/track10/<timestamp>/`
|
||||
|
||||
Place:
|
||||
- `voice-latency-latest.json`
|
||||
- command output log
|
||||
- note with latest call IDs included in report
|
||||
|
||||
## 4) Decision guidance
|
||||
|
||||
GO candidate:
|
||||
- `active_no_end = 0`
|
||||
- `ended_no_recording_upload = 0`
|
||||
- direct p95 metrics within Track 10 thresholds
|
||||
|
||||
NO-GO candidate:
|
||||
- repeated high latency outside thresholds,
|
||||
- stale active calls without `call.ended`,
|
||||
- missing recordings after completed calls.
|
||||
@@ -1,76 +0,0 @@
|
||||
# Runbook - Track 11 Live Operator Call Control
|
||||
|
||||
Use this runbook to verify the accepted call-control path introduced in Track 11.
|
||||
|
||||
For the current operator UX and active/recent call behavior, use `docs/runbooks/track12-operator-voice-ux.md`.
|
||||
|
||||
## Preconditions
|
||||
|
||||
- Track 9 bridge works: `/proxy/asterisk-bridge/asterisk/status` returns `ami_connected=true`
|
||||
- operator SIP extensions exist in Asterisk: `2001`, `2002`
|
||||
- bridge env includes:
|
||||
- `ASTERISK_CALLCONTROL_ENABLED=1`
|
||||
- `ASTERISK_OPERATOR_EXTENSION_MAP_JSON`
|
||||
- `ASTERISK_TRANSFER_TARGET_MAP_JSON`
|
||||
|
||||
## 1) Start the QA stack
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\start_track9_qa.ps1
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```powershell
|
||||
curl http://127.0.0.1:8080/proxy/asterisk-bridge/asterisk/status
|
||||
```
|
||||
|
||||
## 2) Real call scenario
|
||||
|
||||
1. Register caller softphone `1001`.
|
||||
2. Register operator softphone `2001`.
|
||||
3. Make call `1001 -> 7000`.
|
||||
4. Open `http://127.0.0.1:8080/operator` as `operator`.
|
||||
5. In `Живые звонки`:
|
||||
- select the active call
|
||||
- click `Принять в работу`
|
||||
6. Speak in `MicroSIP/Zoiper`.
|
||||
7. Finish with one of:
|
||||
- `Завершить`
|
||||
- `Передать`
|
||||
|
||||
## 3) Verify persisted events
|
||||
|
||||
Expected event types for the same call:
|
||||
- `call.started`
|
||||
- `call.connected`
|
||||
- `call.ended`
|
||||
- `recording.ready`
|
||||
- optional `call.transferred`
|
||||
|
||||
## 4) Verify recording playback
|
||||
|
||||
1. Open `http://127.0.0.1:8080/supervisor`.
|
||||
2. In `Записи`, load the latest recording for the call.
|
||||
3. Check inline playback: `audio/*`, HTTP `200`.
|
||||
|
||||
## 5) Inspect action log
|
||||
|
||||
```powershell
|
||||
curl -H "X-User: operator_a" -H "X-Role: operator" ^
|
||||
http://127.0.0.1:8080/proxy/asterisk-bridge/asterisk/live-calls/<call_id>/actions
|
||||
```
|
||||
|
||||
Each action should have:
|
||||
- `action_type` in `claim|hangup|blind-transfer`
|
||||
- `result_status=ok`
|
||||
- `ami_action_id` for a successful AMI call
|
||||
|
||||
## 6) Failure handling
|
||||
|
||||
- `403 Operator can control only own claimed call`
|
||||
- use the correct operator account or claim the call first
|
||||
- `409 Unable to resolve active channel for call`
|
||||
- check AMI permissions and channel resolution
|
||||
- `Unknown transfer queue_code`
|
||||
- fix `ASTERISK_TRANSFER_TARGET_MAP_JSON`
|
||||
@@ -1,53 +0,0 @@
|
||||
# Runbook - Track 12 Operator Voice UX
|
||||
|
||||
Use this runbook to validate the frozen operator voice baseline after Track 12.
|
||||
|
||||
## Preconditions
|
||||
|
||||
- Platform is running locally or in QA
|
||||
- `ASTERISK_CALLCONTROL_ENABLED=1`
|
||||
- operator softphone is registered: `2001` or `2002`
|
||||
- caller softphone `1001` can call `7000`
|
||||
|
||||
## Operator model
|
||||
|
||||
- The phone answer happens in `MicroSIP/Zoiper`
|
||||
- Browser action `Принять в работу` does not answer media
|
||||
- Browser actions manage ownership and call-control in the platform
|
||||
- `Активные звонки` shows only live bridge-backed calls
|
||||
- `Только что завершённые` shows short-lived recent terminal calls
|
||||
|
||||
## Happy path
|
||||
|
||||
1. Open `/operator` and log in as `operator / op12345`.
|
||||
2. Place a call `1001 -> 7000`.
|
||||
3. Answer in `MicroSIP/Zoiper`.
|
||||
4. Confirm the card appears in `Активные звонки`.
|
||||
5. Click `Принять в работу`.
|
||||
6. Confirm the card stays in `Активные звонки`.
|
||||
7. Click `Завершить`.
|
||||
8. Confirm the card moves to `Только что завершённые`.
|
||||
9. Open `/supervisor` and verify the imported recording.
|
||||
|
||||
## Transfer path
|
||||
|
||||
1. Repeat the call flow.
|
||||
2. Click `Принять в работу`.
|
||||
3. Set:
|
||||
- `target type = extension`
|
||||
- `target value = 2002`
|
||||
4. Click `Передать`.
|
||||
5. Confirm the card moves to `Только что завершённые`.
|
||||
6. Confirm the recent card shows transfer as the terminal action.
|
||||
|
||||
## Passive path
|
||||
|
||||
1. Place a call and answer it in SIP.
|
||||
2. Do not click any browser action for at least 30-40 seconds.
|
||||
3. Confirm the call stays in `Активные звонки` until the real hangup.
|
||||
|
||||
## Failure cues
|
||||
|
||||
- If `Живые звонки` shows `interaction:int_*`, the browser is using stale assets
|
||||
- If a call disappears before the real hangup, inspect bridge reconciliation behavior
|
||||
- If `Передать` succeeds in SIP but not in UI, inspect `/asterisk/live-calls/{call_id}/actions`
|
||||
@@ -1,69 +0,0 @@
|
||||
# Track 14 Browser Softphone Runbook
|
||||
|
||||
## Preconditions
|
||||
|
||||
- `ASTERISK_WEBRTC_ENABLED=1`
|
||||
- `ASTERISK_WEBRTC_WS_URL` points to the Asterisk WSS endpoint
|
||||
- `ASTERISK_BROWSER_SIP_MAP_JSON` contains a mapping for the operator app user
|
||||
- Asterisk `http.conf`, `rtp.conf`, and `pjsip.conf` from `deployment/asterisk/` are applied on the lab VM
|
||||
- The operator workstation trusts the QA certificate used by Asterisk WSS
|
||||
- Chrome or Edge is used for `/operator`
|
||||
|
||||
## Browser config contract
|
||||
|
||||
Bridge route:
|
||||
- `/proxy/asterisk-bridge/asterisk/browser-softphone/config`
|
||||
|
||||
Expected config fields:
|
||||
- `enabled`
|
||||
- `ws_url`
|
||||
- `sip_uri`
|
||||
- `authorization_username`
|
||||
- `password`
|
||||
- `display_name`
|
||||
- `ice_servers`
|
||||
- `operator_extension`
|
||||
|
||||
If the logged-in app user has no browser SIP mapping, the route returns `403`.
|
||||
|
||||
## QA startup flow
|
||||
|
||||
1. Open `/operator` in Chrome or Edge.
|
||||
2. Log in with a user that has:
|
||||
- operator role access
|
||||
- a browser SIP mapping in `ASTERISK_BROWSER_SIP_MAP_JSON`
|
||||
3. Click `Подключить browser softphone`.
|
||||
4. Allow microphone access.
|
||||
5. Confirm the `Browser Softphone` block reaches `Зарегистрирован`.
|
||||
|
||||
## Inbound QA call flow
|
||||
|
||||
1. Place a lab call from `1001` to `7000`.
|
||||
2. Wait for the incoming browser banner.
|
||||
3. Click `Answer`.
|
||||
4. The browser tries to auto-claim the live call.
|
||||
5. If auto-claim does not happen because matching is ambiguous, click `Принять в работу` manually.
|
||||
6. Click `Hang up` in the browser block or `Завершить` in the live-call block.
|
||||
7. Confirm the call moves to `Только что завершённые`.
|
||||
8. Confirm the recording is available in `/supervisor`.
|
||||
|
||||
## Coexistence notes
|
||||
|
||||
- `MicroSIP/Zoiper` remains supported in QA.
|
||||
- Browser softphone does not replace the accepted Track 11/12 control-path.
|
||||
- Browser `Hang up` still uses the accepted bridge hangup route when a live call match exists.
|
||||
|
||||
## Common failures
|
||||
|
||||
### Browser softphone config missing
|
||||
- Check `ASTERISK_BROWSER_SIP_MAP_JSON`
|
||||
- Check that the app user name matches the mapping key exactly
|
||||
|
||||
### WSS registration fails
|
||||
- Check Asterisk `http.conf`
|
||||
- Check that the workstation trusts the QA certificate
|
||||
- Check `ASTERISK_WEBRTC_WS_URL`
|
||||
|
||||
### Browser answer works but auto-claim does not
|
||||
- The current rule requires exactly one active ringing call for the operator extension
|
||||
- If more than one candidate exists, use manual `Принять в работу`
|
||||
@@ -1,44 +0,0 @@
|
||||
# Track 15 - Telegram Chat Runbook
|
||||
|
||||
## Required env
|
||||
|
||||
- `TELEGRAM_BOT_ENABLED=1`
|
||||
- `TELEGRAM_BOT_TOKEN`
|
||||
- `TELEGRAM_WEBHOOK_SECRET`
|
||||
- `TELEGRAM_DEFAULT_QUEUE_ID`
|
||||
|
||||
Optional:
|
||||
|
||||
- `TELEGRAM_BOT_API_BASE` when running against a stub or proxy
|
||||
|
||||
## Register the bot webhook
|
||||
|
||||
Use Telegram Bot API `setWebhook` with:
|
||||
|
||||
- URL -> your public `telegram-adapter-service` webhook path:
|
||||
- `/integrations/telegram/bot/webhook`
|
||||
- secret token -> `TELEGRAM_WEBHOOK_SECRET`
|
||||
|
||||
## Operator workflow
|
||||
|
||||
1. Open `/operator`
|
||||
2. Go to the `Telegram` page
|
||||
3. Wait for a real inbound bot message
|
||||
4. Select the thread from the left list
|
||||
5. Click `Принять в работу`
|
||||
6. Reply from the composer
|
||||
7. Close or escalate the thread if needed
|
||||
|
||||
## Runtime behavior
|
||||
|
||||
- one Telegram thread is stored per `chat_id`
|
||||
- new inbound messages reuse the same `thread_id` and `interaction_id`
|
||||
- if the linked interaction was `closed`, a new inbound message reactivates it to `new`
|
||||
- unsupported inbound Telegram content is stored as a visible system placeholder
|
||||
|
||||
## QA check
|
||||
|
||||
- send one first message from Telegram -> thread appears
|
||||
- send a second message from the same `chat_id` -> same thread is reused
|
||||
- operator claim works
|
||||
- operator reply appears in the thread and is sent via Bot API
|
||||
@@ -1,134 +0,0 @@
|
||||
# Runbook - Track 7 Scale Validation
|
||||
|
||||
## Goal
|
||||
|
||||
Validate the production-like `scale500` profile for Wave 2 / Track 7 on `K8s on-prem`.
|
||||
|
||||
This runbook assumes:
|
||||
|
||||
- Helm is the canonical deployment path
|
||||
- `DATABASE_URL` points to a shared PostgreSQL endpoint
|
||||
- `metrics-server` is installed and healthy
|
||||
- Helm migration hook is enabled
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `kubectl` access to the target cluster
|
||||
- `helm` installed locally
|
||||
- namespace chosen (examples below use `mvp-cc`)
|
||||
- Helm chart configured with:
|
||||
- `global.schemaManagementMode=migrations`
|
||||
- `migrations.enabled=true`
|
||||
|
||||
## Configure Helm values
|
||||
|
||||
Before deployment, confirm:
|
||||
|
||||
- `profiles.scale500.enabled=true`
|
||||
- `global.databaseUrl` is set to PostgreSQL
|
||||
- `global.schemaManagementMode=migrations`
|
||||
- `migrations.enabled=true`
|
||||
- `db.poolSize`, `db.maxOverflow`, `db.poolTimeoutSeconds`, `db.poolRecycleSeconds` match the target baseline
|
||||
|
||||
Render and inspect:
|
||||
|
||||
```powershell
|
||||
helm lint deployment\helm
|
||||
helm template mvp-cc deployment\helm -n mvp-cc
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
```powershell
|
||||
helm upgrade --install mvp-cc deployment\helm -n mvp-cc --create-namespace
|
||||
```
|
||||
|
||||
Wait until:
|
||||
|
||||
- migration Job `mvp-cc-db-migrate` completed successfully
|
||||
- all expected pods are `Ready`
|
||||
- HPA objects are created for the hot-path services
|
||||
- `recording-service` stays singleton
|
||||
|
||||
## Step validation (`250 + 250`)
|
||||
|
||||
Run the staged profile first:
|
||||
|
||||
```powershell
|
||||
python scripts\load_test.py --base-url http://<gateway-host> --profile step_250_250 --auth-mode bearer
|
||||
python scripts\track7_check.py --namespace mvp-cc --report-dir .artifacts\track7\<timestamp> --require-success-rate 99 --require-p95-seconds 1.5 --require-p99-seconds 3.0
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- success rate `>= 99%`
|
||||
- no crash loops or OOM kills
|
||||
- no unexpected pod restarts
|
||||
|
||||
You can also run the staged helper, which writes a formal acceptance pack automatically:
|
||||
|
||||
```powershell
|
||||
python scripts\track7_stage.py --base-url http://<gateway-host> --namespace mvp-cc --auth-mode bearer
|
||||
```
|
||||
|
||||
This creates:
|
||||
|
||||
- `.artifacts/track7/staged_<timestamp>/step_250_250`
|
||||
- `.artifacts/track7/staged_<timestamp>/target_500_500`
|
||||
- `.artifacts/track7/staged_<timestamp>/acceptance_summary.json`
|
||||
- `.artifacts/track7/staged_<timestamp>/acceptance_summary.md`
|
||||
|
||||
## Target validation (`500 + 500`)
|
||||
|
||||
Only after the step profile passes:
|
||||
|
||||
```powershell
|
||||
python scripts\load_test.py --base-url http://<gateway-host> --profile target_500_500 --auth-mode bearer
|
||||
python scripts\track7_check.py --namespace mvp-cc --report-dir .artifacts\track7\<timestamp> --require-success-rate 99 --require-p95-seconds 2.0 --require-p99-seconds 3.5
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `500 voice + 500 digital` mixed profile
|
||||
- success rate `>= 99%`
|
||||
- p95 `<= 2.0s`
|
||||
- p99 `<= 3.5s`
|
||||
- `5xx` / transport failure rate `<= 0.5%`
|
||||
|
||||
## Evidence pack
|
||||
|
||||
Keep the following together:
|
||||
|
||||
- rendered Helm values/profile
|
||||
- `summary.json`
|
||||
- `latency_samples.csv`
|
||||
- `error_samples.json`
|
||||
- `mix_profile.json`
|
||||
- output from `track7_check.py`
|
||||
- rollout notes (cluster, namespace, date, profile, result)
|
||||
|
||||
## Rollback
|
||||
|
||||
If the target profile fails:
|
||||
|
||||
1. Roll back the Helm release:
|
||||
|
||||
```powershell
|
||||
helm rollback mvp-cc
|
||||
```
|
||||
|
||||
2. Keep the failed report directory for analysis.
|
||||
3. Reduce to the last passing profile and re-run validation only after tuning.
|
||||
|
||||
## Canonical scale500 values
|
||||
|
||||
Use the built-in preset as the base overlay for the production-like scale profile:
|
||||
|
||||
```powershell
|
||||
helm upgrade --install mvp-cc deployment\helm -f deployment\helm\values.scale500.yaml -n mvp-cc --create-namespace
|
||||
```
|
||||
|
||||
Then override at least:
|
||||
|
||||
- `global.databaseUrl`
|
||||
- image tag / repository as needed for the target cluster
|
||||
@@ -1,190 +0,0 @@
|
||||
# Wave 2 / Track 9.2 - Production Cutover (K8s Helm, Controlled Window)
|
||||
|
||||
This runbook executes Track 9.2 as an operations cutover, not a feature track.
|
||||
|
||||
## Goal
|
||||
|
||||
Move the accepted Asterisk bridge flow from QA to production Helm deployment with:
|
||||
|
||||
- strict bridge auth (`bearer` only),
|
||||
- one real softphone call validation,
|
||||
- recording import + supervisor playback proof,
|
||||
- formal GO/NO-GO evidence and rollback path.
|
||||
|
||||
## Scope
|
||||
|
||||
Includes:
|
||||
|
||||
- production Helm upgrade in strict mode,
|
||||
- post-cutover checks:
|
||||
- `track9_preflight --check-sftp --require-strict-service-auth`
|
||||
- `asterisk_lab_smoke --require-recording`
|
||||
- `track9_check --require-recording`
|
||||
- evidence package collection.
|
||||
|
||||
Excludes:
|
||||
|
||||
- ARI/call-control,
|
||||
- new voice features,
|
||||
- business API contract changes.
|
||||
|
||||
## Required inputs
|
||||
|
||||
Fill [track9-cutover-sheet.template.md](/e:/Zhan/docs/acceptance/track9/track9-cutover-sheet.template.md) first.
|
||||
|
||||
Mandatory values:
|
||||
|
||||
- `KUBE_CONTEXT`
|
||||
- `NAMESPACE`
|
||||
- `RELEASE`
|
||||
- `GATEWAY_BASE_URL`
|
||||
- `DATABASE_URL` (PostgreSQL in production)
|
||||
- `IMAGE_TAG`
|
||||
- `ASTERISK_AMI_HOST`, `ASTERISK_AMI_USERNAME`, `ASTERISK_AMI_SECRET`
|
||||
- `ASTERISK_SFTP_HOST`, `ASTERISK_SFTP_USERNAME`, `ASTERISK_SFTP_PASSWORD`
|
||||
- `QUEUE_ID` for `voice_lab`
|
||||
- `APP_TOKEN_SECRET`
|
||||
|
||||
## Phase 0 - Entry checks (T-1 day)
|
||||
|
||||
1. Baseline reference is fixed:
|
||||
- [docs/acceptance/track9/track9-acceptance.md](/e:/Zhan/docs/acceptance/track9/track9-acceptance.md)
|
||||
2. Confirm `voice_lab -> queue_id` exists in target routing DB.
|
||||
3. Confirm cluster can reach Asterisk VM:
|
||||
- AMI `5038/tcp`
|
||||
- SFTP `22/tcp`
|
||||
4. Confirm maintenance window and owners.
|
||||
|
||||
## Phase 1 - Pre-cutover dry-run (T-4h to T-1h)
|
||||
|
||||
Run dry-run command (no deploy):
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\track9_2_cutover.ps1 `
|
||||
-KubeContext <KUBE_CONTEXT> `
|
||||
-Namespace <NAMESPACE> `
|
||||
-Release <RELEASE> `
|
||||
-GatewayBaseUrl <GATEWAY_BASE_URL> `
|
||||
-DatabaseUrl <DATABASE_URL> `
|
||||
-ImageTag <IMAGE_TAG> `
|
||||
-AmiHost <ASTERISK_AMI_HOST> `
|
||||
-AmiUser <ASTERISK_AMI_USERNAME> `
|
||||
-AmiSecret <ASTERISK_AMI_SECRET> `
|
||||
-SftpHost <ASTERISK_SFTP_HOST> `
|
||||
-SftpUser <ASTERISK_SFTP_USERNAME> `
|
||||
-SftpPassword <ASTERISK_SFTP_PASSWORD> `
|
||||
-QueueId <QUEUE_ID> `
|
||||
-AppTokenSecret <APP_TOKEN_SECRET>
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `helm lint` PASS
|
||||
- `helm template` PASS
|
||||
- rendered manifest contains:
|
||||
- `ALLOW_LEGACY_HEADER_AUTH=0`
|
||||
- `ASTERISK_BRIDGE_AUTH_MODE=bearer`
|
||||
- `ASTERISK_BRIDGE_AUTH_FALLBACK_LEGACY=0`
|
||||
- `VOICE_ADAPTER_TRUSTED_SERVICE_SUBJECTS` with `svc:asterisk-bridge`
|
||||
- `RECORDING_IMPORT_TRUSTED_SERVICE_SUBJECTS` with `svc:asterisk-bridge`
|
||||
- `RECORDING_IMPORT_ALLOW_ADMIN=0`
|
||||
|
||||
Artifacts are saved under:
|
||||
|
||||
- `.artifacts/track9_2/<timestamp>/`
|
||||
|
||||
## Phase 2 - Controlled cutover execute (T0)
|
||||
|
||||
Run same command with `-Execute`:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\track9_2_cutover.ps1 `
|
||||
-KubeContext <KUBE_CONTEXT> `
|
||||
-Namespace <NAMESPACE> `
|
||||
-Release <RELEASE> `
|
||||
-GatewayBaseUrl <GATEWAY_BASE_URL> `
|
||||
-DatabaseUrl <DATABASE_URL> `
|
||||
-ImageTag <IMAGE_TAG> `
|
||||
-AmiHost <ASTERISK_AMI_HOST> `
|
||||
-AmiUser <ASTERISK_AMI_USERNAME> `
|
||||
-AmiSecret <ASTERISK_AMI_SECRET> `
|
||||
-SftpHost <ASTERISK_SFTP_HOST> `
|
||||
-SftpUser <ASTERISK_SFTP_USERNAME> `
|
||||
-SftpPassword <ASTERISK_SFTP_PASSWORD> `
|
||||
-QueueId <QUEUE_ID> `
|
||||
-AppTokenSecret <APP_TOKEN_SECRET> `
|
||||
-Execute
|
||||
```
|
||||
|
||||
If Helm reports conflicts on HPA-managed `Deployment.spec.replicas`, rerun with:
|
||||
|
||||
```powershell
|
||||
... -Execute -ForceUpgrade
|
||||
```
|
||||
|
||||
Script actions:
|
||||
|
||||
1. snapshot:
|
||||
- `kubectl get pods -o wide`
|
||||
- `helm history` (before)
|
||||
2. `helm upgrade --install` with strict overlay
|
||||
3. wait for the migration Job to complete
|
||||
4. rollout wait:
|
||||
- `api-gateway`
|
||||
- `asterisk-bridge-service`
|
||||
- `voice-adapter-service`
|
||||
- `recording-service`
|
||||
5. strict preflight + smoke + track9_check
|
||||
6. evidence package via `track9_collect_evidence`
|
||||
7. writes rollback hint from Helm history
|
||||
|
||||
## Phase 3 - Live acceptance (mandatory)
|
||||
|
||||
During window, place one real call:
|
||||
|
||||
- softphone `1001 -> 7000`
|
||||
|
||||
Then confirm:
|
||||
|
||||
- `call.started` from Asterisk source exists
|
||||
- `call.ended` for same call exists
|
||||
- `recording.ready` uploaded to `recording-service`
|
||||
- supervisor playback is `HTTP 200` and `audio/*`
|
||||
|
||||
## Phase 4 - Evidence and GO/NO-GO
|
||||
|
||||
Use generated evidence folder from script and finalize:
|
||||
|
||||
- `track9-acceptance.md`
|
||||
- `playback-proof.md`
|
||||
- optional `playback-proof.json`
|
||||
|
||||
GO only if all true:
|
||||
|
||||
1. preflight PASS
|
||||
2. smoke PASS
|
||||
3. track9_check PASS
|
||||
4. `failed bridge events = 0`
|
||||
5. supervisor playback confirmed
|
||||
|
||||
## Rollback policy
|
||||
|
||||
Rollback triggers:
|
||||
|
||||
- preflight FAIL
|
||||
- track9_check FAIL
|
||||
- no recording import after live call
|
||||
- failed bridge events keep growing
|
||||
|
||||
Rollback command pattern:
|
||||
|
||||
```powershell
|
||||
helm -n <NAMESPACE> rollback <RELEASE> <PREVIOUS_REVISION>
|
||||
kubectl -n <NAMESPACE> rollout status deploy/api-gateway
|
||||
kubectl -n <NAMESPACE> rollout status deploy/asterisk-bridge-service
|
||||
```
|
||||
|
||||
After rollback:
|
||||
|
||||
- rerun `track9_preflight` in previous baseline,
|
||||
- log incident note + root cause item in cutover sheet.
|
||||
@@ -1,49 +0,0 @@
|
||||
# Voice Baseline
|
||||
|
||||
Status: `Track 11/12 accepted, Track 14 browser QA path added`
|
||||
|
||||
This document describes what the voice path already supports today.
|
||||
|
||||
## What works
|
||||
|
||||
- Asterisk 20 on a Linux VM forwards inbound lab calls into the platform through `asterisk-bridge-service`
|
||||
- A call from `1001` to `7000` creates a real voice interaction and writes:
|
||||
- `call.started`
|
||||
- `call.connected`
|
||||
- `call.ended`
|
||||
- `recording.ready`
|
||||
- `/operator` shows real bridge-backed live calls in `Активные звонки`
|
||||
- The operator can use accepted browser controls:
|
||||
- `Принять в работу`
|
||||
- `Завершить`
|
||||
- `Передать`
|
||||
- `/operator` keeps short-lived terminal cards in `Только что завершённые`
|
||||
- Recording import is automatic and `/supervisor` can play the audio back
|
||||
- A QA browser softphone path is now available when:
|
||||
- bridge WebRTC config is enabled
|
||||
- the operator user has a browser SIP mapping
|
||||
- Asterisk WSS config is applied
|
||||
|
||||
## What this baseline does not do
|
||||
|
||||
- no ARI
|
||||
- no attended transfer or conference
|
||||
- no production cutover workflow
|
||||
- no telephony redesign beyond the accepted Track 11 and Track 12 behavior
|
||||
- no production browser softphone rollout yet
|
||||
|
||||
## Source documents
|
||||
|
||||
- Track 11 roadmap: `docs/roadmap/07-track11-live-operator-call-control.md`
|
||||
- Track 12 roadmap: `docs/roadmap/08-track12-operator-voice-ux-hardening.md`
|
||||
- Track 14 roadmap: `docs/roadmap/09-track14-browser-softphone-webrtc.md`
|
||||
- Track 11 runbook: `docs/runbooks/track11-live-operator-call-control.md`
|
||||
- Track 12 runbook: `docs/runbooks/track12-operator-voice-ux.md`
|
||||
- Track 14 runbook: `docs/runbooks/track14-browser-softphone.md`
|
||||
|
||||
## Acceptance references
|
||||
|
||||
- Track 11 accepted package:
|
||||
- `docs/acceptance/track11/track11-acceptance.md`
|
||||
- Track 12 accepted package:
|
||||
- `docs/acceptance/track12/20260307_161629/track12-acceptance.md`
|
||||
Reference in New Issue
Block a user