sync: migrate ai-operator to Gitea (2026-08-10)

2026-08-10 15:26:52 +00:00
commit 53652b95ad
173 changed files with 16676 additions and 0 deletions
@@ -0,0 +1,81 @@
+# Agent Prompt and Tools
+
+TZ-10 adds the final AI-operator behavior layer on top of the existing dialogue state machine, language selection, region resolver, Knowledge Base, and VoiceProvider abstraction.
+
+## Behavior
+
+The operator must:
+
+- introduce itself as Жанна, the AI operator of QazAimaqGas;
+- avoid IVR-style "choose language" and "choose region" prompts;
+- infer Russian or Kazakh from the caller's speech;
+- ask for region only when a regional answer is needed;
+- answer general questions through global KB without selected region;
+- call only tools allowed by the current dialogue state;
+- answer business questions only after `search_knowledge_base`;
+- use only KB tool results as answer evidence;
+- return a controlled no-answer message when KB has no relevant result;
+- format the caller-facing answer in the selected language;
+- support `kk` answers from `ru` KB sources through cross-language fallback without inventing facts.
+
+## Final Prompt
+
+`internal/agent.BuildSystemPrompt` builds the state-aware system prompt. It includes Жанна/QazAimaqGas identity, current state, language, region, allowed next actions, forbidden actions, no-hallucination rules, and internal-detail restrictions.
+
+The live flow starts ready to listen after the natural greeting. `LANGUAGE_SELECTION` and `REGION_SELECTION` remain for compatibility, but they are no longer blocking IVR steps. In `READY_TO_HELP` and `QUESTION_ANSWERING`, the assistant must call `search_knowledge_base` before answering business questions.
+
+If region is missing:
+
+- general questions search `global` KB;
+- branch, address, contact, city, oblast, or region questions return `region_required_for_question`;
+- "Алматы" remains ambiguous and must be clarified as city vs oblast.
+
+## Tool Flow
+
+```text
+VoiceProvider event
+  -> Call Manager voice event pump
+  -> Dialogue Orchestrator
+  -> Tool Policy
+  -> KB Service when allowed
+  -> ToolResult
+  -> VoiceProvider.SendToolResult
+```
+
+Tool denials are returned as structured tool results and are safe to send back to the provider.
+
+## Knowledge Answers
+
+`search_knowledge_base` returns `answer_text`, `answer_language`, KB results, citations, `cross_language_fallback_used`, `message_key`, and `reason_code`.
+
+The answer text is built from returned KB content only. It does not expose chunk IDs, SQL, embeddings, vector search, credentials, or prompt internals.
+
+## Cross-Language Fallback
+
+If the caller selected `kk` and no `kk` KB result exists, the KB service can search `ru` content. TZ-10 formats the response with Kazakh framing while keeping factual content grounded in the returned source.
+
+## CLI
+
+```bash
+./bin/ai-operator agent-self-test --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator natural-dialogue-self-test --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator openai-tool-smoke-test --env /etc/ai-operator/ai-operator.env
+```
+
+`agent-self-test` and `natural-dialogue-self-test` use PostgreSQL, fake embeddings, and FakeVoiceProvider. They do not call OpenAI. `openai-tool-smoke-test` fails before network when `OPENAI_API_KEY` is empty.
+
+## Not Implemented Yet
+
+- real business tools;
+- human transfer;
+- final production route switch;
+- production Kazakhtelecom call testing;
+- curated Kazakh KB content parity.
+## TZ-11 Handoff Prompt Rules
+
+If the caller asks for a human, operator, consultant, specialist, or live agent, the assistant must call `request_human_handoff`. It must not promise a real transfer unless the tool result confirms successful transfer. If the tool result is stubbed or not configured, the assistant must explain that politely.
+
+
+## Audit Integration
+
+Tool calls and tool results are persisted through `ai_tool_audit` after recursive redaction. KB search audit records result counts and citations, not raw credentials or audio.
@@ -0,0 +1,73 @@
+# Architecture
+
+Target architecture:
+
+```text
+Kazakhtelecom SIP
+  -> Asterisk / PJSIP
+  -> inbound dialplan
+  -> Stasis(ai-operator)
+  -> Go AI Operator
+  -> ARI Call Manager
+  -> chan_websocket Media Gateway
+  -> Voice Provider Adapter
+  -> OpenAI Realtime or future provider
+  -> Knowledge Base / RAG
+```
+
+TZ-01 status:
+
+- Stasis route is not enabled.
+- Media gateway is not implemented.
+- OpenAI provider is not implemented.
+- Knowledge Base / RAG is not implemented.
+- Live call handling is disabled.
+
+Created package boundaries:
+
+- `internal/asterisk/ari`: local ARI health-check client.
+- `internal/call`: call session, language, and state types.
+- `internal/media`: media gateway interfaces and audio chunk types.
+- `internal/ai`: voice provider interfaces and events.
+- `internal/dialogue`: orchestrator interface and skeleton stub.
+- `internal/knowledge`: search repository interface.
+- `internal/tools`: future tool names.
+
+## TZ-03 status
+
+The Go service can register ARI app `ai-operator`, read ARI events, parse Stasis and hangup lifecycle events, and manage in-memory sessions. Media Gateway, OpenAI provider, and production route switching are still not implemented.
+
+## TZ-04 status
+
+The service can create ARI externalMedia WebSocket channels, connect to `/media/{connection_id}`, parse media control events, send media control commands, and collect basic media stats. OpenAI and production routing remain disabled.
+
+## TZ-05 status
+
+The service now has a replaceable VoiceProvider layer with a fake provider and OpenAI Realtime skeleton. OpenAI remains behind an adapter. Conversation policy, language/region flow, tools execution, and RAG are not implemented yet.
+
+## TZ-06 Conversation Layer
+
+Call Manager now delegates dialogue state to `internal/dialogue`. The state machine enforces language and region selection before any knowledge-base or business-tool flow. Production routing remains unchanged.
+
+## TZ-07 Language Selection
+
+`internal/dialogue/language` normalizes user text, detects Russian/Kazakh choices, and applies state-aware selection policy. It feeds the Dialogue Orchestrator, which stores strict `ru`/`kk` only.
+
+## TZ-08 Region Resolver Update
+
+TZ-08 adds an offline Kazakhstan Region Resolver with 20 enabled regions, stable `region_code` values, Almaty city/region clarification, false-positive protection, and explicit region-change policy. Production routing remains disabled; OpenAI, PostgreSQL, and Knowledge Base/RAG are still not used at this stage.
+
+## TZ-09 Knowledge Base Update
+
+TZ-09 adds PostgreSQL + pgvector, KGA JSONL ingestion, fake embeddings, hybrid KB search, cross-language fallback for KK queries, citations, and `search_knowledge_base` integration behind the existing language/region guardrail. OpenAI is not called automatically and production routing remains disabled.
+## TZ-10 Agent Prompt and Tools
+
+The agent behavior layer sits above Dialogue Orchestrator and below VoiceProvider. It builds the final state-aware prompt, formats KB-grounded answers, preserves no-hallucination behavior, and relays VoiceProvider ToolCall results back through `SendToolResult`.
+## TZ-11 Human Handoff / Fallback
+
+The handoff layer sits beside Dialogue Orchestrator. User requests for an operator are detected from transcript text or tool calls, converted into structured handoff requests, policy-gated, and executed by a safe executor. Default mode is `disabled_stub`.
+
+
+## TZ-12 Audit Layer
+
+The audit layer records call lifecycle, state transitions, redacted transcript events, tool calls, KB searches, handoff/fallback events, provider/media stats, exports and retention dry-runs through `internal/audit`.
@@ -0,0 +1,60 @@
+# ARI Call Manager
+
+## Purpose
+
+The ARI Call Manager connects the Go service to Asterisk ARI events for application `ai-operator`. It is responsible for reading ARI WebSocket events, creating in-memory call sessions, and performing limited test-call control.
+
+## Modes
+
+### Observe-only
+
+```sh
+./bin/ai-operator run --env /etc/ai-operator/ai-operator.env --ari-events --observe-only
+```
+
+Observe-only registers the ARI app and reads events. It does not answer, hang up, create bridges, create media channels, or call OpenAI.
+
+### Call-control
+
+```sh
+./bin/ai-operator run --env /etc/ai-operator/ai-operator.env --ari-events --call-control --test-call-hangup-after 5s
+```
+
+Call-control is restricted to Stasis args containing `test`. It may answer a test channel and hang it up after the configured delay. Production routes are disabled by default.
+
+## Implemented in TZ-03
+
+- ARI events WebSocket listener.
+- Basic Auth and query `api_key` URL support.
+- Safe sanitized WebSocket URL builder.
+- Reconnect with bounded backoff.
+- `ApplicationReplaced` handling.
+- Lock file protection for one active listener.
+- `StasisStart`, `StasisEnd`, `ChannelHangupRequest`, `ChannelDestroyed`, and generic event parsing.
+- In-memory thread-safe session store.
+- ARI HTTP actions for answer, hangup, and get channel.
+
+## Not Implemented Yet
+
+- Media gateway.
+- Bridges.
+- `chan_websocket` media channel handling.
+- OpenAI provider.
+- Knowledge base / RAG.
+- Production route switching.
+
+## Safety Notes
+
+- Only one ARI listener should run at a time.
+- The default lock file is `/tmp/ai-operator-ari-events.lock`.
+- Production route handling is disabled.
+- Test route is `7199@from-softphones` and enters `Stasis(ai-operator,test)`.
+- Kazakhtelecom production routes are not switched.
+
+## TZ-04 media integration
+
+Call-control mode can optionally enable media with `--media`. Media is restricted to the `test` route and uses the Asterisk WebSocket Media Gateway. Production routes remain disabled.
+
+## TZ-05 Voice Provider Integration
+
+Call-control test mode can start a VoiceProvider session. This is restricted to the `test` route and production remains disabled.
@@ -0,0 +1,90 @@
+# Asterisk Routing
+
+## Current State
+
+Production Kazakhtelecom inbound routing remains unchanged.
+
+Current production contexts:
+
+- `from-telecom`
+- `from-telecom-sales`
+
+Both still route to the existing AudioSocket entrypoint:
+
+```text
+from-softphones,7100,1
+```
+
+The current AudioSocket route is preserved as the rollback path. It uses the existing local AudioSocket service and `slin16` media path.
+
+A test-only Stasis route has been added for future controlled ARI testing. The Stasis app name is:
+
+```text
+ai-operator
+```
+
+## Test Route
+
+Context:
+
+```text
+ai-operator-test
+```
+
+Extension:
+
+```text
+7199
+```
+
+Softphone entrypoint:
+
+```text
+7199@from-softphones
+```
+
+Dialplan behavior:
+
+```text
+Stasis(ai-operator,test)
+```
+
+The route does not call `Answer()`. The future Go ARI Call Manager is expected to answer the channel.
+
+Purpose:
+
+- controlled test route for TZ-03 ARI Call Manager;
+- controlled test route for TZ-04 Media Gateway;
+- not a production Kazakhtelecom route.
+
+Warning: do not call this route before the Go ARI service is ready except for a controlled expected-failure test. Without an active ARI app, Asterisk may reject or end the call path.
+
+## Future Production Switch
+
+Switch production Kazakhtelecom routes to `Stasis(ai-operator,production,...)` only after:
+
+- TZ-03 ARI Call Manager is implemented;
+- TZ-04 Media Gateway is implemented;
+- a controlled softphone test succeeds;
+- rollback has been confirmed;
+- Kazakhtelecom SIP registration no-response is resolved or production testing is otherwise approved.
+
+Do not switch `from-telecom` or `from-telecom-sales` during skeleton or unit-test work.
+
+## Rollback
+
+Rollback target:
+
+```text
+Goto(from-softphones,7100,1)
+```
+
+Rollback procedure:
+
+1. Restore the AudioSocket route for the affected production DID or pattern.
+2. Remove or disable the production `Stasis(ai-operator,production,...)` route.
+3. Confirm there are no active calls.
+4. Run `dialplan reload`.
+5. Verify `7100@from-softphones` still exists.
+
+Reference patches are stored outside the repository in `/tmp/ai-operator-tz02` for this TZ.
@@ -0,0 +1,120 @@
+# Conversation State Machine
+
+## Purpose
+
+The conversation state machine enforces the safe order of an AI Operator call. The assistant cannot answer business questions, search a knowledge base, or execute business tools until both language and region are selected in code.
+
+## States
+
+```text
+CALL_STARTED
+  -> GREETING
+  -> LANGUAGE_SELECTION
+  -> REGION_SELECTION
+  -> READY_TO_HELP
+  -> QUESTION_ANSWERING
+  -> READY_TO_HELP
+  -> CLOSING
+  -> ENDED
+```
+
+`HANDOFF` can be entered from any non-ended state. `CLOSING` and `ENDED` can also be entered from any non-ended state. `ENDED` is immutable.
+
+## Transition Table
+
+| From | Event | To | Requirement |
+| --- | --- | --- | --- |
+| CALL_STARTED | call.started | GREETING | call session exists |
+| GREETING | greeting.played | LANGUAGE_SELECTION | bilingual greeting prepared |
+| LANGUAGE_SELECTION | language.selected | REGION_SELECTION | language is `ru` or `kk` |
+| REGION_SELECTION | region.selected | READY_TO_HELP | normalized region code is selected |
+| READY_TO_HELP | question.received | QUESTION_ANSWERING | language and region selected |
+| QUESTION_ANSWERING | answer.completed | READY_TO_HELP | answer flow completed |
+| any non-ended | handoff.requested | HANDOFF | none |
+| any non-ended | closing.requested | CLOSING | none |
+| any non-ended | call.ended | ENDED | none |
+
+Invalid transitions return errors and do not mutate state. Denied actions are recorded in session history.
+
+## Tool Permission Matrix
+
+| State | Allowed Tools |
+| --- | --- |
+| CALL_STARTED | none |
+| GREETING | none |
+| LANGUAGE_SELECTION | `set_language`, `request_human_handoff`, `end_call` |
+| REGION_SELECTION | `set_region`, `set_language`, `request_human_handoff`, `end_call` |
+| READY_TO_HELP | `search_knowledge_base`, `set_language`, `set_region`, `request_human_handoff`, `end_call` |
+| QUESTION_ANSWERING | `search_knowledge_base`, `request_human_handoff`, `end_call`, `set_language`, `set_region` |
+| HANDOFF | `end_call` |
+| CLOSING | `end_call` |
+| ENDED | none |
+
+## Hard Guardrail
+
+`search_knowledge_base` is denied unless all of these are true:
+
+- language is `ru` or `kk`;
+- region status is `selected`;
+- region code is not empty;
+- state is `READY_TO_HELP` or `QUESTION_ANSWERING`.
+
+This is enforced in Go code by `internal/dialogue/policy`, independently of prompts.
+
+## Messages
+
+The message catalog has bilingual initial greeting plus Russian and Kazakh fragments for language, region, ready, denial, handoff, and closing states.
+
+Initial greeting:
+
+```text
+Здравствуйте! Это AI-оператор. Выберите язык обслуживания: русский или қазақша.
+Сәлеметсіз бе! Бұл AI-оператор. Қызмет көрсету тілін таңдаңыз: қазақша немесе русский.
+```
+
+## Prompt Fragments
+
+`internal/dialogue/prompt` builds state-aware prompt fragments that include current state, language, region, next action, and forbidden actions. Prompt fragments are secondary; policy enforcement remains in Go code.
+
+## CLI Usage
+
+```bash
+./bin/ai-operator doctor --env /etc/ai-operator/ai-operator.env --check-dialogue
+./bin/ai-operator dialogue-self-test --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator dialogue-self-test --env /etc/ai-operator/ai-operator.env --json
+```
+
+The self-test runs without Asterisk calls, OpenAI, database, or external services.
+
+## Integration
+
+For the test route only, Call Manager creates a Dialogue Orchestrator session before starting the voice provider. The provider receives a state-aware system prompt. On hangup, Call Manager ends and cleans the dialogue session.
+
+Production Kazakhtelecom routes remain disabled.
+
+## Not Implemented Yet
+
+- language detection from speech;
+- region resolver;
+- Knowledge Base/RAG;
+- final agent prompt;
+- real business tools;
+- production route switch.
+
+## TZ-07 Language Detector
+
+Language selection is now handled by `internal/dialogue/language`. The state machine still stores strict `ru` or `kk`, while detector input can be natural text such as `русский`, `қазақша`, `russian`, or `kazakh`.
+
+## TZ-08 Region Resolver Update
+
+TZ-08 adds an offline Kazakhstan Region Resolver with 20 enabled regions, stable `region_code` values, Almaty city/region clarification, false-positive protection, and explicit region-change policy. Production routing remains disabled; OpenAI, PostgreSQL, and Knowledge Base/RAG are still not used at this stage.
+
+## TZ-09 Knowledge Base Update
+
+TZ-09 adds PostgreSQL + pgvector, KGA JSONL ingestion, fake embeddings, hybrid KB search, cross-language fallback for KK queries, citations, and `search_knowledge_base` integration behind the existing language/region guardrail. OpenAI is not called automatically and production routing remains disabled.
+## TZ-10 Agent Integration
+
+The state machine remains the hard gate for business answers. Agent prompt instructions are secondary; `search_knowledge_base` is still denied before language and region are selected. ToolCall results now flow back to the VoiceProvider.
+## TZ-11 Handoff
+
+`request_human_handoff` moves active conversations to `HANDOFF`. Repeated handoff requests are idempotent. `ENDED` remains immutable.
@@ -0,0 +1,153 @@
+# Environment
+
+Default env file:
+
+```text
+/etc/ai-operator/ai-operator.env
+```
+
+Priority:
+
+1. CLI `--env`.
+2. `AI_OPERATOR_ENV_FILE`.
+3. `/etc/ai-operator/ai-operator.env`.
+
+Required:
+
+- `APP_ENV`: `dev`, `staging`, or `prod`.
+- `LOG_LEVEL`: `debug`, `info`, `warn`, or `error`.
+- `ASTERISK_ARI_URL`: must be localhost HTTP.
+- `ASTERISK_ARI_WS_URL`: must be localhost WebSocket.
+- `ASTERISK_ARI_USER`.
+- `ASTERISK_ARI_PASSWORD`.
+- `ASTERISK_ARI_APP`.
+- `ASTERISK_ARI_WS_AUTH_MODE`: optional, defaults to `basic`; allowed `basic`, `query_api_key`, `auto`.
+- `ASTERISK_MEDIA_MODE`: expected `chan_websocket` for MVP.
+- `ASTERISK_MEDIA_CODEC`: expected `slin16` for MVP.
+
+Optional for now:
+
+- `OPENAI_API_KEY`.
+- `OPENAI_REALTIME_MODEL`.
+- `OPENAI_REALTIME_INITIAL_GREETING`: defaults to `true`; live OpenAI calls create an audible natural Жанна greeting after session setup.
+- `DATABASE_URL`.
+
+Example without real secrets:
+
+```text
+APP_ENV=dev
+LOG_LEVEL=debug
+ASTERISK_ARI_URL=http://127.0.0.1:8088/ari
+ASTERISK_ARI_WS_URL=ws://127.0.0.1:8088/ari/events
+ASTERISK_ARI_USER=ai_operator
+ASTERISK_ARI_PASSWORD=CHANGE_ME
+ASTERISK_ARI_APP=ai-operator
+ASTERISK_ARI_WS_AUTH_MODE=basic
+ASTERISK_MEDIA_MODE=chan_websocket
+ASTERISK_MEDIA_CODEC=slin16
+OPENAI_API_KEY=
+OPENAI_REALTIME_MODEL=
+DATABASE_URL=
+```
+
+## Voice Provider / OpenAI
+
+`VOICE_PROVIDER` defaults to `fake`. Allowed values are `fake`, `openai_realtime`, `pipeline_elevenlabs`, and `pipeline_elevenlabs_streaming`. `OPENAI_API_KEY` is optional for build, tests, and doctor. It is required only for live OpenAI modes and explicit smoke commands.
+
+## ElevenLabs Streaming Pipeline
+
+The low-latency live pipeline uses `pipeline_elevenlabs_streaming`:
+
+```env
+STT_PROVIDER=elevenlabs
+ELEVENLABS_STT_MODE=realtime
+ELEVENLABS_STT_MODEL=scribe_realtime_v2
+ELEVENLABS_STT_LANGUAGE_AUTO=true
+ELEVENLABS_STT_SAMPLE_RATE=16000
+ELEVENLABS_STT_INPUT_FORMAT=pcm_16000
+ELEVENLABS_STT_PARTIAL_ENABLED=true
+ELEVENLABS_STT_COMMITTED_ONLY_FOR_LLM=true
+
+LLM_PROVIDER=openai
+LLM_MODEL=gpt-4.1-nano
+LLM_STREAM=true
+LLM_FIRST_CHUNK_TIMEOUT_MS=1200
+LLM_STREAM_CHUNK_MIN_CHARS=30
+LLM_STREAM_CHUNK_MAX_CHARS=160
+
+ELEVENLABS_API_KEY=
+ELEVENLABS_VOICE_ID_RU=
+ELEVENLABS_VOICE_ID_KK=
+ELEVENLABS_TTS_MODE=websocket
+ELEVENLABS_TTS_MODEL_ID=eleven_flash_v2_5
+ELEVENLABS_TTS_OUTPUT_FORMAT=pcm_16000
+ELEVENLABS_TTS_SAMPLE_RATE=16000
+
+VOICE_NATURALNESS_ENABLED=true
+VOICE_AUDIO_TAGS_ENABLED=true
+VOICE_AUDIO_TAGS_MODE=controlled
+VOICE_ALLOW_COUGH=false
+
+PIPELINE_INITIAL_GREETING=true
+PIPELINE_BARGE_IN=true
+PIPELINE_FLUSH_ON_USER_SPEECH=true
+PIPELINE_TTS_START_AFTER_CHARS=60
+PIPELINE_TTS_START_AFTER_PUNCTUATION=true
+```
+
+Live `pipeline_elevenlabs_streaming` requires `ELEVENLABS_API_KEY`, `ELEVENLABS_VOICE_ID_RU`, and `OPENAI_API_KEY`. If `ELEVENLABS_VOICE_ID_KK` is empty, the RU voice is used for Kazakh with a sanitized warning.
+
+## Dialogue Variables
+
+```env
+DIALOGUE_DEFAULT_LANGUAGE=
+DIALOGUE_DEFAULT_REGION_CODE=
+DIALOGUE_ENFORCE_LANGUAGE_REGION=false
+AGENT_NAME=Жанна
+AGENT_COMPANY=QazAimaqGas
+AGENT_INITIAL_GREETING_MODE=natural
+AGENT_REQUIRE_EXPLICIT_LANGUAGE=false
+AGENT_REQUIRE_REGION_BEFORE_HELP=false
+AGENT_ALLOW_GLOBAL_KB_WITHOUT_REGION=true
+```
+
+The live operator now uses a natural Жанна/QazAimaqGas flow: language is inferred from speech, global KB can be searched without a selected region, and region is requested only for regional questions.
+
+## Language Selection Variables
+
+```env
+LANGUAGE_SELECTION_ENABLED=false
+LANGUAGE_CONFIDENCE_THRESHOLD=0.70
+LANGUAGE_CHANGE_CONFIDENCE_THRESHOLD=0.90
+LANGUAGE_ALLOW_CHANGE_AFTER_SELECTION=true
+LANGUAGE_DEFAULT_ON_TIMEOUT=
+```
+
+Classic IVR-style language selection is disabled for live dialogue. Explicit language-change phrases are still supported.
+
+## TZ-08 Region Resolver Update
+
+TZ-08 adds an offline Kazakhstan Region Resolver with 20 enabled regions, stable `region_code` values, Almaty city/region clarification, false-positive protection, and explicit region-change policy. Production routing remains disabled; OpenAI, PostgreSQL, and Knowledge Base/RAG are still not used at this stage.
+
+## TZ-09 Knowledge Base Update
+
+TZ-09 adds PostgreSQL + pgvector, KGA JSONL ingestion, fake embeddings, hybrid KB search, cross-language fallback for KK queries, citations, and `search_knowledge_base` integration. In natural dialogue mode, global KB search is allowed without a selected region; regional branch/address/contact questions ask for region first. OpenAI is not called automatically and production routing remains disabled.
+## TZ-10 Agent Prompt and Tools
+
+No new required environment variables were added in TZ-10. `DATABASE_URL` is required for `agent-self-test` because it validates real KB search. `OPENAI_API_KEY` is only required for explicit OpenAI smoke commands and is not needed for fake-provider validation.
+## TZ-11 Handoff / Fallback
+
+Default handoff settings are safe and do not transfer calls:
+
+```env
+HANDOFF_MODE=disabled_stub
+HANDOFF_ENABLED=false
+HANDOFF_ALLOW_IN_TEST_ROUTE_ONLY=true
+```
+
+Real transfer modes require explicit target configuration and must remain disabled until approved.
+
+
+## Audit Environment
+
+TZ-12 adds `AUDIT_*` retention, transcript storage and redaction settings. Defaults are safe: audit enabled, PostgreSQL sink, fail-open, raw transcripts disabled, redacted transcripts enabled, full phone/IIN/card logging disabled.
@@ -0,0 +1,134 @@
+# Human Handoff / Fallback
+
+TZ-11 adds a safe Human Handoff and fallback layer. The default behavior is intentionally non-transferring.
+
+## Purpose
+
+Human handoff gives the AI operator a controlled way to react when the caller asks for a live operator or when automated service cannot safely continue.
+
+## Default Behavior
+
+Default configuration:
+
+```env
+HANDOFF_MODE=disabled_stub
+HANDOFF_ENABLED=false
+HANDOFF_ALLOW_IN_TEST_ROUTE_ONLY=true
+```
+
+With these defaults:
+
+- no real transfer is attempted;
+- `request_human_handoff` creates a structured request;
+- conversation state moves to `HANDOFF`;
+- the caller receives a safe message that direct transfer is not connected yet.
+
+## Handoff Modes
+
+- `disabled_stub`: records the request and returns a safe message. No ARI transfer.
+- `ari_redirect`: uses ARI channel redirect when explicitly enabled and configured.
+- `dialplan_continue`: returns the channel to dialplan when explicitly enabled and configured.
+- `hangup_after_message`: returns a final message and moves toward closing without transfer.
+
+Real transfer modes require `HANDOFF_ENABLED=true` and valid target fields. Production routes remain protected.
+
+## Config Variables
+
+- `HANDOFF_MODE`
+- `HANDOFF_ENABLED`
+- `HANDOFF_TARGET_ENDPOINT`
+- `HANDOFF_DIALPLAN_CONTEXT`
+- `HANDOFF_DIALPLAN_EXTENSION`
+- `HANDOFF_DIALPLAN_PRIORITY`
+- `HANDOFF_QUEUE_NAME`
+- `HANDOFF_TIMEOUT`
+- `HANDOFF_MAX_ATTEMPTS`
+- `HANDOFF_PLAY_MESSAGE_BEFORE_TRANSFER`
+- `HANDOFF_HANGUP_AFTER_STUB`
+- `HANDOFF_ALLOW_IN_TEST_ROUTE_ONLY`
+- `HANDOFF_MAX_SUMMARY_CHARS`
+
+Fallback thresholds:
+
+- `FALLBACK_MAX_LANGUAGE_FAILURES`
+- `FALLBACK_MAX_REGION_FAILURES`
+- `FALLBACK_MAX_NO_ANSWER`
+- `FALLBACK_MAX_KB_UNAVAILABLE`
+- `FALLBACK_MAX_AI_ERRORS`
+- `FALLBACK_MAX_MEDIA_ERRORS`
+- `FALLBACK_MAX_TOOL_ERRORS`
+- `FALLBACK_CALL_TIMEOUT`
+
+## Detector Phrases
+
+The handoff detector supports Russian, Kazakh, and mixed/Latin phrases such as:
+
+- `оператор`
+- `соедините с оператором`
+- `хочу поговорить с человеком`
+- `операторға қосыңыз`
+- `маман керек`
+- `live agent`
+- `speak to human`
+
+Ambiguous business phrases such as `оператор связи` are not treated as confident handoff requests.
+
+## Fallback Scenarios
+
+Fallback manager can offer handoff or safe closing for:
+
+- KB unavailable;
+- repeated no-answer;
+- repeated language failures;
+- repeated region failures;
+- AI provider error;
+- media error;
+- tool error;
+- call timeout.
+
+## ARI Redirect Mode
+
+`ari_redirect` calls:
+
+```text
+POST /ari/channels/{channelId}/redirect?endpoint={endpoint}
+```
+
+It is only used when enabled and configured. Required tests use fake/httptest clients.
+
+## Dialplan Continue Mode
+
+`dialplan_continue` calls:
+
+```text
+POST /ari/channels/{channelId}/continue?context={context}&extension={extension}&priority={priority}
+```
+
+It is only used when enabled and configured.
+
+## Production Route
+
+Production routes remain disabled. `HANDOFF_ALLOW_IN_TEST_ROUTE_ONLY=true` prevents real transfer outside the test route.
+
+## Safe Tests
+
+```bash
+./bin/ai-operator doctor --env /etc/ai-operator/ai-operator.env --check-handoff
+./bin/ai-operator handoff-self-test --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator fallback-self-test --env /etc/ai-operator/ai-operator.env
+```
+
+These commands do not perform real transfer.
+
+## Not Implemented Yet
+
+- real operator queue/extension;
+- production handoff enablement;
+- call transcript persistence;
+- CRM ticket creation;
+- operator schedule/availability checks.
+
+
+## Audit Integration
+
+Handoff requests write `ai_handoff_audit` records with redacted summaries and transfer status. Default `disabled_stub` does not attempt real transfer.
@@ -0,0 +1,134 @@
+# Knowledge Base / RAG
+
+## Purpose
+
+TZ-09 adds a PostgreSQL + pgvector backed Knowledge Base / RAG layer for the AI Operator. The initial production-like data source is the KGA JSONL import under `/opt/ai-operator/knowledge/import/jsonl`.
+
+## Architecture
+
+The KB layer contains:
+
+- PostgreSQL with pgvector, bound to `127.0.0.1` only.
+- `knowledge_documents` for source JSONL records.
+- `knowledge_chunks` for searchable text chunks and embeddings.
+- `knowledge_ingest_runs` for import accounting.
+- `knowledge_search_logs` for sanitized search metadata.
+- EmbeddingProvider abstraction with `fake` default and OpenAI skeleton.
+- Hybrid search: vector score, keyword/full-text score, and region boost.
+
+## Deployment
+
+Docker Compose file:
+
+```bash
+/opt/ai-operator/deploy/docker-compose.postgres.yml
+```
+
+Postgres env file:
+
+```bash
+/etc/ai-operator/postgres.env
+```
+
+The env file is root-owned and mode `600`. The database port is bound as:
+
+```text
+127.0.0.1:${AI_OPERATOR_POSTGRES_PORT:-5432}:5432
+```
+
+## Source Files
+
+Required KGA JSONL files:
+
+```text
+/opt/ai-operator/knowledge/import/jsonl/kga_kb_rag.jsonl
+/opt/ai-operator/knowledge/import/jsonl/kga_branches_by_region.jsonl
+```
+
+`kga_kb_rag.jsonl` contains the main FAQ/RAG records. `kga_branches_by_region.jsonl` contains regional branch contacts.
+
+Required fields are `external_id`, `title`, `language`, `region_code`, `status`, `content`, and `source_hash`. Unknown fields are preserved in document metadata.
+
+## Chunking
+
+The chunker is UTF-8 safe and uses deterministic character windows:
+
+- target: 1200 chars
+- max: 2000 chars
+- overlap: 150 chars
+
+Chunk text keeps title, question, short answer, full answer, keywords, source, and content context.
+
+## Embeddings
+
+Default provider: `fake`.
+
+The fake provider is deterministic, local-only, 1536-dimensional, and suitable for tests/MVP plumbing. The OpenAI provider is implemented as a low-level HTTP skeleton but is not called automatically.
+
+## Search
+
+Search filters:
+
+- `status='published'`
+- `valid_from` / `valid_to`
+- selected language first
+- selected region plus `global`
+
+For Kazakh (`kk`), if no KK result exists, cross-language fallback searches Russian (`ru`) and marks returned results with `cross_language_fallback=true`.
+
+Ranking:
+
+```text
+score = 0.65 * vector_score + 0.25 * keyword_score + 0.10 * region_boost
+```
+
+Region boost is `1.0` for exact region and `0.6` for global fallback.
+
+## Tool Integration
+
+`search_knowledge_base` is only allowed after language and region are selected and the conversation state is `READY_TO_HELP` or `QUESTION_ANSWERING`.
+
+If no result exists, the tool returns:
+
+```json
+{"ok": false, "reason_code": "no_relevant_knowledge", "message_key": "knowledge.no_answer"}
+```
+
+The dialogue layer does not generate a final natural-language answer in TZ-09. TZ-10 will use the returned chunks and citations to produce user-facing answers.
+
+## CLI
+
+```bash
+./bin/ai-operator kb-migrate --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator kb-health --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator kb-ingest --env /etc/ai-operator/ai-operator.env --path /opt/ai-operator/knowledge/import/jsonl --format jsonl --embedding-provider fake
+./bin/ai-operator kb-search --env /etc/ai-operator/ai-operator.env --query "Сколько стоит первичное подключение газа?" --language ru --region-code almaty_city --limit 5
+./bin/ai-operator kb-self-test --env /etc/ai-operator/ai-operator.env
+```
+
+## Safety
+
+- No OpenAI call is made unless an explicit live embedding command is run with an API key.
+- DB credentials are masked in logs and reports.
+- No raw audio or base64 audio is stored.
+- No Asterisk configs, dialplan, SIP, ARI, AMI, or RTP configs are changed.
+- Production routing remains disabled.
+
+## Not Implemented Yet
+
+- final agent prompt
+- real business tools
+- human handoff transfer
+- web/admin UI
+- document approval workflow
+## TZ-10 Tool Answer Payload
+
+`search_knowledge_base` now returns caller-facing `answer_text` in addition to KB results and citations. The answer text is generated only from returned KB content. Cross-language fallback can return Russian source content for a Kazakh caller, with Kazakh framing and no unsupported facts.
+## TZ-11 Fallback
+
+If KB is unavailable or repeated searches return no relevant answer, fallback can offer human handoff. The AI must still not invent an answer.
+
+
+## Audit Integration
+
+KB searches write `ai_kb_audit` rows containing redacted query text, language, region, result count, top score, citation count and cross-language fallback status.
@@ -0,0 +1,81 @@
+# Language Selection
+
+## Purpose
+
+Language Selection recognizes an explicit customer choice between Russian (`ru`) and Kazakh (`kk`) before business help is allowed. It is implemented as `internal/dialogue/language` and is independent from Asterisk, OpenAI, and the media pipeline.
+
+## Supported Languages
+
+- `ru`: Russian
+- `kk`: Kazakh
+
+The stored language is always one of these strict internal codes.
+
+## Detection Pipeline
+
+1. Normalize text: trim, lowercase, normalize `ё` to `е`, collapse spaces, remove harmless punctuation, preserve Kazakh Cyrillic letters.
+2. Match exact codes and phrase tables.
+3. Detect mixed RU/KK and ethnicity ambiguity.
+4. Apply confidence and reason codes.
+5. Run selection policy based on conversation state.
+
+## Russian Phrases
+
+Examples include `ru`, `rus`, `русский`, `русский язык`, `на русском`, `по-русски`, `хочу на русском`, `говорите на русском`, `russian`, `speak russian`, `russki`, and ASR-like variants such as `руский`, `русски`, `порусски`.
+
+## Kazakh Phrases
+
+Examples include `kk`, `kz` as alias to `kk`, `қазақша`, `қазақ тілі`, `қазақ тілінде`, `қазақша сөйлейік`, `казахский`, `на казахском`, `qazaqsha`, `kazakh`, `kazaksha`, and ASR-like variants such as `қазакша`, `казакша`, `показахски`.
+
+## Ambiguous Cases
+
+The detector asks for clarification for cases such as:
+
+- `я русский`
+- `я казах`
+- `русский или қазақша?`
+- `сначала русский потом казахский`
+
+These are not applied as automatic language selection.
+
+## False-Positive Protection
+
+The detector avoids language selection for terms such as `Казахтелеком`, `русский клиент спрашивает`, `У меня вопрос на русском сайте`, embedded `ru`/`kk` substrings, and ordinary business questions.
+
+## Language Change Rules
+
+- In `LANGUAGE_SELECTION`, clear RU/KK choices are applied.
+- In `REGION_SELECTION`, language changes require a clear phrase.
+- In `READY_TO_HELP` and `QUESTION_ANSWERING`, language changes require explicit switch intent.
+- In `HANDOFF`, `CLOSING`, and `ENDED`, language changes are denied.
+
+## Integration
+
+The Dialogue Orchestrator exposes `HandleUserText`. It passes transcripts or CLI input through the detector, applies selection policy, then calls the same `set_language` transition path. `VoiceEventUserTranscriptDone` is wired to this path. Assistant transcript/audio events do not trigger language detection.
+
+## CLI
+
+```bash
+./bin/ai-operator doctor --env /etc/ai-operator/ai-operator.env --check-language
+./bin/ai-operator language-self-test --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator language-self-test --env /etc/ai-operator/ai-operator.env --json
+```
+
+## Not Implemented Yet
+
+- Region Resolver
+- Knowledge Base/RAG
+- final business prompt
+- production routing
+- real ASR accuracy validation
+
+## TZ-08 Region Resolver Update
+
+TZ-08 adds an offline Kazakhstan Region Resolver with 20 enabled regions, stable `region_code` values, Almaty city/region clarification, false-positive protection, and explicit region-change policy. Production routing remains disabled; OpenAI, PostgreSQL, and Knowledge Base/RAG are still not used at this stage.
+
+## TZ-09 Knowledge Base Update
+
+TZ-09 adds PostgreSQL + pgvector, KGA JSONL ingestion, fake embeddings, hybrid KB search, cross-language fallback for KK queries, citations, and `search_knowledge_base` integration behind the existing language/region guardrail. OpenAI is not called automatically and production routing remains disabled.
+## TZ-10 Agent Integration
+
+The agent prompt keeps language selection as the first required action. Business questions in `LANGUAGE_SELECTION` must repeat the language request and must not trigger KB search.
@@ -0,0 +1,85 @@
+# Logging, Transcripts and Audit
+
+## Purpose
+
+TZ-12 adds a PostgreSQL audit trail for AI Operator calls. The audit layer records lifecycle events, state transitions, redacted transcripts, tool calls, KB searches, handoff/fallback outcomes, provider/media events, and retention runs.
+
+## Architecture
+
+Audit is enabled by default with `AUDIT_SINK=postgres`. Runtime writes are best-effort by default: `AUDIT_FAIL_CLOSED=false` means a temporary audit write failure is logged in sanitized form and does not break the call.
+
+The code is split into:
+
+- `internal/audit`: repository, service, export and retention logic.
+- `internal/audit/redaction`: deterministic redaction for text, JSON and byte payloads.
+- `migrations/003_audit_tables.sql`: audit schema.
+
+## Audit Tables
+
+- `ai_calls`: call-level record with masked caller number.
+- `ai_call_events`: lifecycle, state, tool, KB, handoff, provider, media and security events.
+- `ai_transcript_events`: redacted transcript events. Raw encrypted text is reserved and remains null by default.
+- `ai_tool_audit`: redacted tool args/results and allow/deny status.
+- `ai_kb_audit`: redacted query, result counts, citations and fallback flags.
+- `ai_handoff_audit`: handoff mode/status and redacted summary/target.
+- `ai_provider_audit`: provider counters and sanitized errors.
+- `ai_media_audit`: media stats only, never raw audio.
+- `ai_audit_retention_runs`: retention dry-run/confirm audit.
+
+## Event Taxonomy
+
+Stable event names include `call.started`, `call.ended`, `conversation.state_transition`, `conversation.denied_action`, `transcript.user.final`, `transcript.assistant.final`, `tool.call.completed`, `kb.search.completed`, `kb.search.no_answer`, `handoff.requested`, `fallback.triggered`, `provider.error`, `media.stats`, and `security.redaction_applied`.
+
+## Transcript Policy
+
+Defaults are privacy-first:
+
+- `AUDIT_STORE_TRANSCRIPTS=true`
+- `AUDIT_STORE_TRANSCRIPT_DELTAS=false`
+- `AUDIT_STORE_RAW_TRANSCRIPTS=false`
+- `AUDIT_STORE_REDACTED_TRANSCRIPTS=true`
+
+Only redacted final transcript text is stored by default. Raw audio and base64 audio are never stored by the audit layer.
+
+## Redaction Rules
+
+The redactor masks phone numbers, IIN-like 12 digit values, card-like 16 digit values, emails, OpenAI/API keys, Bearer tokens, password/secret/api_key values, PostgreSQL URL passwords, OTP/SMS codes in context, and account-like numbers. Recursive JSON redaction is used for tool args/results and metadata.
+
+## Retention Policy
+
+Defaults:
+
+- audit events: 180 days
+- transcripts: 30 days
+- tool audit: 180 days
+- error audit: 365 days
+
+`audit-prune --dry-run` is the default safe mode and records a retention run without deleting rows.
+
+## CLI
+
+```bash
+./bin/ai-operator audit-health --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator redaction-self-test --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator audit-self-test --env /etc/ai-operator/ai-operator.env
+./bin/ai-operator audit-show-call --env /etc/ai-operator/ai-operator.env --call-id CALL_ID
+./bin/ai-operator audit-export-call --env /etc/ai-operator/ai-operator.env --call-id CALL_ID --output /tmp/call-export.json
+./bin/ai-operator audit-prune --env /etc/ai-operator/ai-operator.env --dry-run
+./bin/ai-operator doctor --env /etc/ai-operator/ai-operator.env --check-audit
+```
+
+## Export Format
+
+Call export is JSON and contains the call row, events, redacted transcripts, tool audit, KB audit, handoff audit, provider audit and media audit. Raw transcript text and secrets are not included.
+
+## Privacy and Security
+
+Do not enable raw transcript storage in production without a separate encryption and approval design. Do not log raw audio, base64 audio, full phone numbers, full IINs, full card numbers, database passwords, API keys, SIP credentials or ARI/AMI secrets.
+
+## Not Implemented Yet
+
+- Admin UI.
+- Encrypted raw transcript storage.
+- External SIEM integration.
+- Real call recordings.
+- Production route switch.
@@ -0,0 +1,82 @@
+# Media Gateway
+
+## Purpose
+
+The Media Gateway connects the Go AI Operator to Asterisk audio through `chan_websocket` media channels.
+
+## Why chan_websocket
+
+Asterisk 20.16.0 on this server has `chan_websocket` and `res_http_websocket` loaded, and HTTP status exposes `/media/...` as Media over Websocket. This avoids RTP socket handling for the MVP media path.
+
+## Asterisk 20.16.0 Compatibility
+
+- Plain-text media control messages are used by default.
+- JSON control parsing is implemented for future compatibility.
+- No dependency on `transport_data=f(json)`.
+- Codec for TZ-04 is `slin16`.
+- Media WebSocket subprotocol is `media`.
+
+## Runtime Flow
+
+```text
+Caller calls 7199@from-softphones
+  -> Stasis(ai-operator,test)
+  -> Go ARI Call Manager receives StasisStart
+  -> Answer caller
+  -> Create mixing bridge
+  -> Add caller channel
+  -> Create externalMedia WebSocket channel
+  -> Get MEDIA_WEBSOCKET_CONNECTION_ID
+  -> Go connects to ws://127.0.0.1:8088/media/{connection_id}
+  -> Add media channel to bridge
+  -> Pump caller slin16 audio to VoiceProvider
+  -> Pump VoiceProvider assistant audio deltas back to Asterisk
+  -> Cleanup
+```
+
+Production Kazakhtelecom routes remain unchanged. AudioSocket `7100` remains the rollback route.
+
+## CLI Usage
+
+Media self-test without real call:
+
+```sh
+./bin/ai-operator media-self-test --env /etc/ai-operator/ai-operator.env --duration 3s
+```
+
+Controlled test route media mode:
+
+```sh
+./bin/ai-operator run --env /etc/ai-operator/ai-operator.env --ari-events --call-control --media --voice-provider openai_realtime --media-test-mode stats --test-call-hangup-after 180s
+```
+
+## Media Test Modes
+
+- `stats`: collect media stats and do not synthesize test audio. Real VoiceProvider audio is still forwarded to the caller.
+- `silence`: send short slin16 silence.
+- `tone`: send a short low-volume sine tone for controlled testing only.
+- `echo`: reserved for controlled testing, not a default mode.
+
+## Safety
+
+- Production route is disabled.
+- No audio recording by default.
+- Raw audio frames are not saved to disk by default.
+- OpenAI is not called.
+- Cleanup is designed to be idempotent.
+
+## Troubleshooting
+
+- No `MEDIA_START`: verify externalMedia channel creation and `/media/{connection_id}` URL.
+- Missing `MEDIA_WEBSOCKET_CONNECTION_ID`: retry variable fetch; channel may not be fully initialized yet.
+- WebSocket 404/401: verify ARI/HTTP localhost binding and credentials.
+- XOFF stuck: outbound audio pauses until `MEDIA_XON` or context timeout.
+- Leaked bridge/channel: run `core show channels` and `bridge show all`, then clean up aiop-prefixed resources.
+
+## Voice Provider Integration
+
+Media Gateway audio is forwarded to a VoiceProvider in test route mode. Caller audio is resampled from Asterisk `slin16` 16 kHz PCM16 to provider 24 kHz PCM16. Assistant audio deltas are resampled back to 16 kHz PCM16 and sent to Asterisk over the media WebSocket. Default provider is `fake`; OpenAI requires explicit `--voice-provider openai_realtime` and an API key.
+
+## TZ-06 Dialogue Safety
+
+Media Gateway remains transport-only. It does not decide business state. Dialogue state and tool permissions are handled by the Dialogue Orchestrator.
@@ -0,0 +1,52 @@
+# Next Steps
+
+TZ-05 implemented the VoiceProvider abstraction, FakeVoiceProvider, and OpenAI Realtime skeleton. The next implementation stage is:
+
+- TZ-06: Conversation State Machine.
+
+Planned sequence:
+
+- TZ-06: Conversation State Machine.
+- TZ-07: Language Selection.
+- TZ-08: Region Resolver.
+- TZ-09: Knowledge Base / RAG.
+- TZ-10: Agent Prompt and Tools.
+
+Do not switch the production inbound route until language/region policy, voice provider behavior, media handling, and rollback are tested end to end.
+
+## After TZ-06
+
+Next: TZ-07 Language Selection. Implement language selection behavior on top of the strict `set_language` tool and state machine. Do not enable production routing yet.
+
+## After TZ-07
+
+Next: TZ-08 Region Resolver. Language is now selected and guarded in code; region still requires a normalized resolver before Knowledge Base work.
+
+## After TZ-08
+
+Next: TZ-09 Knowledge Base / RAG. Language and region are now normalized and guarded in code; `search_knowledge_base` remains a stub until the KB layer is implemented.
+
+## After TZ-09
+
+Next: TZ-10 Agent Prompt and Tools. The KB/RAG layer can now return chunks and citations; TZ-10 should turn those results into guarded user-facing answers and tool behavior.
+## After TZ-10
+
+Current completed stage: ТЗ-10: Agent Prompt and Tools.
+
+Next: ТЗ-11: Human Handoff.
+## After TZ-11
+
+Current completed stage: ТЗ-11: Human Handoff / Fallback.
+
+Next: ТЗ-12: Logging, Transcripts and Audit.
+
+
+## Next
+
+ТЗ-13: Security Hardening.
+
+## After TZ-12
+
+Current completed stage: ТЗ-12: Logging, Transcripts and Audit.
+
+Next: ТЗ-13: Security Hardening.
@@ -0,0 +1,106 @@
+# Region Resolver
+
+## Purpose
+
+TZ-08 adds an offline Kazakhstan Region Resolver for the dialogue layer. It normalizes user text, maps Russian/Kazakh/Latin aliases to stable `region_code` values, and prevents the conversation from reaching `READY_TO_HELP` until a supported region is selected.
+
+The resolver does not call external APIs, does not infer region from caller number, and does not use default region fallback.
+
+## Supported Active Regions
+
+Active catalog size: 20 regions.
+
+Republic cities:
+
+- `astana_city` - Астана / Астана
+- `almaty_city` - город Алматы / Алматы қаласы
+- `shymkent_city` - Шымкент / Шымкент
+
+Oblasts:
+
+- `akmola_region` - Акмолинская область / Ақмола облысы
+- `aktobe_region` - Актюбинская область / Ақтөбе облысы
+- `almaty_region` - Алматинская область / Алматы облысы
+- `atyrau_region` - Атырауская область / Атырау облысы
+- `east_kazakhstan_region` - Восточно-Казахстанская область / Шығыс Қазақстан облысы
+- `zhambyl_region` - Жамбылская область / Жамбыл облысы
+- `west_kazakhstan_region` - Западно-Казахстанская область / Батыс Қазақстан облысы
+- `karaganda_region` - Карагандинская область / Қарағанды облысы
+- `kostanay_region` - Костанайская область / Қостанай облысы
+- `kyzylorda_region` - Кызылординская область / Қызылорда облысы
+- `mangystau_region` - Мангистауская область / Маңғыстау облысы
+- `pavlodar_region` - Павлодарская область / Павлодар облысы
+- `north_kazakhstan_region` - Северо-Казахстанская область / Солтүстік Қазақстан облысы
+- `turkistan_region` - Туркестанская область / Түркістан облысы
+- `abai_region` - область Абай / Абай облысы
+- `jetisu_region` - область Жетісу / Жетісу облысы
+- `ulytau_region` - область Ұлытау / Ұлытау облысы
+
+Optional disabled special region:
+
+- `baikonur_special` - Байконур / Байқоңыр, disabled by default.
+
+## Aliases And Normalization
+
+The resolver supports RU, KK, Latin, abbreviations, legacy names, and ASR-ish spellings. Examples include `нурсултан`, `нұр-сұлтан`, `alma-ata`, `shymkent`, `chimkent`, `ВКО`, `ЗКО`, `СКО`, `vko`, `zko`, and `sko`.
+
+Normalization trims spaces, lowercases, converts `ё` to `е`, treats hyphens as spaces, collapses repeated spaces, removes harmless punctuation, preserves Kazakh Cyrillic letters, and handles forms like `г. Алматы`, `Алматы қаласы`, and `Алматинская обл.`.
+
+## Ambiguity
+
+Bare `Алматы` and `almaty` are intentionally ambiguous because they can mean the city or Almaty region. The orchestrator stores pending candidates and asks:
+
+- RU: `Вы имеете в виду город Алматы или Алматинскую область?`
+- KK: `Алматы қаласын айттыңыз ба, әлде Алматы облысын ба?`
+
+Then these clarifications are accepted:
+
+- `город`, `қала`, `Алматы қаласы` -> `almaty_city`
+- `область`, `облыс`, `Алматинская область`, `Алматы облысы` -> `almaty_region`
+
+## Selection Policy
+
+- `LANGUAGE_SELECTION`: region mentions are ignored; language must be selected first.
+- `REGION_SELECTION`: clear enabled region applies; ambiguous input asks clarification; unsupported region is denied.
+- `READY_TO_HELP` and `QUESTION_ANSWERING`: region changes only on explicit request such as `сменить регион на Астану`.
+- `HANDOFF`, `CLOSING`, `ENDED`: region changes are denied.
+
+The hard guardrail remains in code: `search_knowledge_base` is denied unless language is `ru` or `kk`, region status is `selected`, `region_code` is non-empty, and state is `READY_TO_HELP` or `QUESTION_ANSWERING`.
+
+## Integration
+
+`Dialogue Orchestrator` calls the resolver from `HandleUserText` and `set_region` tool handling. Stored region display names come from the catalog, not from tool arguments.
+
+`VoiceEventUserTranscriptDone` continues to route through `HandleUserText`, so future ASR transcripts use the same resolver path. Assistant transcript and audio events do not trigger region changes.
+
+## CLI
+
+Run resolver checks:
+
+```bash
+./bin/ai-operator doctor --env /etc/ai-operator/ai-operator.env --check-region
+```
+
+Run deterministic self-test without Asterisk, OpenAI, or DB:
+
+```bash
+./bin/ai-operator region-self-test --env /etc/ai-operator/ai-operator.env
+```
+
+## Safety
+
+TZ-08 does not change Asterisk configs, does not reload/restart Asterisk, does not call OpenAI, does not install PostgreSQL, and does not switch production routing.
+
+## Not Implemented Yet
+
+- Knowledge Base / RAG
+- final business prompt
+- real business tools
+- production route switching
+
+## TZ-09 Knowledge Base Update
+
+TZ-09 adds PostgreSQL + pgvector, KGA JSONL ingestion, fake embeddings, hybrid KB search, cross-language fallback for KK queries, citations, and `search_knowledge_base` integration behind the existing language/region guardrail. OpenAI is not called automatically and production routing remains disabled.
+## TZ-10 Agent Integration
+
+The agent prompt keeps region selection as the second required action. Business questions in `REGION_SELECTION` must repeat the region request and must not trigger KB search. Region changes after readiness remain explicit-only.
@@ -0,0 +1,47 @@
+# Security
+
+- ARI must listen only on localhost.
+- AMI must listen only on localhost.
+- Never log `ASTERISK_ARI_PASSWORD`.
+- Never log OpenAI API keys.
+- Mask database URL credentials before logging.
+- Keep `/etc/ai-operator/ai-operator.env` owned by root with mode `600`.
+- Do not commit `.env` files.
+- Do not modify SIP credentials as part of the Go application skeleton.
+- Do not store OpenAI keys in the repository.
+
+## OpenAI Safety
+
+Do not log `OPENAI_API_KEY`, Authorization headers, raw audio, or base64 audio payloads. OpenAI may only be called by explicit smoke-test or explicit `--voice-provider openai_realtime` test mode.
+
+## Dialogue Guardrails
+
+Knowledge-base search and future business tools are denied in code until language and region are selected. This must remain enforced outside the prompt.
+
+## Language Selection Safety
+
+The detector does not call external services and does not log raw audio. Language changes after selection require explicit user intent to avoid accidental switching.
+
+## TZ-08 Region Resolver Update
+
+TZ-08 adds an offline Kazakhstan Region Resolver with 20 enabled regions, stable `region_code` values, Almaty city/region clarification, false-positive protection, and explicit region-change policy. Production routing remains disabled; OpenAI, PostgreSQL, and Knowledge Base/RAG are still not used at this stage.
+
+## TZ-09 Knowledge Base Update
+
+TZ-09 adds PostgreSQL + pgvector, KGA JSONL ingestion, fake embeddings, hybrid KB search, cross-language fallback for KK queries, citations, and `search_knowledge_base` integration behind the existing language/region guardrail. OpenAI is not called automatically and production routing remains disabled.
+## Agent Prompt and Tools
+
+- The agent prompt forbids revealing chunk IDs, SQL, vector search, embeddings, credentials, and internal prompts to callers.
+- Business answers must be grounded in `search_knowledge_base` results only.
+- Tool denials are structured and safe to return to the VoiceProvider.
+## Human Handoff
+
+- Real transfer is disabled by default.
+- Handoff summaries are sanitized and must not contain full phone numbers.
+- ARI redirect and dialplan continue are config-gated and test-route protected.
+- ARI credentials and transfer targets with credentials must not be logged.
+
+
+## TZ-12 Audit Security
+
+Audit persistence redacts phone numbers, IINs, cards, emails, OTP codes, API keys, Bearer tokens, database URL passwords and recursive JSON values. Raw audio and base64 audio are not stored.
@@ -0,0 +1,21 @@
+# TZ Map
+
+- TZ-00: Server, Asterisk, SIP, ARI, and media audit.
+- TZ-00.5: Prerequisite hardening and server preparation.
+- TZ-01: Go project architecture and skeleton.
+- TZ-02: Asterisk inbound routing preparation.
+- TZ-03: ARI Call Manager.
+- TZ-04: chan_websocket Media Gateway.
+- TZ-05: Voice Provider Adapter.
+- TZ-06: Conversation State Machine.
+- TZ-07: Language Selection.
+- TZ-08: Region Resolver.
+- TZ-09: Knowledge Base / RAG.
+- TZ-10: Tool execution layer.
+- TZ-11: Persistence and call history.
+- TZ-12: Observability and metrics.
+- TZ-13: Deployment hardening.
+- TZ-14: End-to-end staging tests.
+- TZ-15: Operator handoff flow.
+- TZ-16: Production readiness.
+- TZ-17: Launch checklist.
@@ -0,0 +1,122 @@
+# Voice Provider
+
+## Purpose
+
+`VoiceProvider` is the replaceable voice AI boundary for the AI Operator. OpenAI is one adapter, not the foundation of the application.
+
+## Providers
+
+- `fake`: offline provider for tests and local integration without external calls.
+- `openai_realtime`: low-level WebSocket adapter for OpenAI Realtime.
+
+## OpenAI Realtime Flow
+
+```text
+connect WebSocket
+  -> session.created
+  -> session.update
+  -> input_audio_buffer.append
+  -> response.output_audio.delta
+  -> response.done / function_call
+```
+
+The adapter builds client events, parses server events, emits normalized `VoiceEvent` values, and never exposes the API key in logs.
+
+## Audio Path
+
+```text
+Asterisk slin16 audio
+  -> PCM16 validation
+  -> optional 16000 <-> 24000 linear resampling
+  -> base64 for OpenAI input_audio_buffer.append
+  -> OpenAI output audio delta
+  -> base64 decode
+  -> optional resample back to Asterisk slin16
+  -> Media Gateway SendAudio
+```
+
+Raw audio and base64 payloads are not logged or saved by default.
+
+## CLI Usage
+
+OpenAI config check without network:
+
+```sh
+./bin/ai-operator doctor --env /etc/ai-operator/ai-operator.env --check-openai-config
+```
+
+Fake provider self-test:
+
+```sh
+./bin/ai-operator voice-provider-self-test --env /etc/ai-operator/ai-operator.env --provider fake
+```
+
+Live OpenAI smoke-test, only when `OPENAI_API_KEY` is configured:
+
+```sh
+./bin/ai-operator openai-smoke-test --env /etc/ai-operator/ai-operator.env
+```
+
+Test route run with fake provider:
+
+```sh
+./bin/ai-operator run --env /etc/ai-operator/ai-operator.env --ari-events --call-control --media --voice-provider fake
+```
+
+Test route run with OpenAI Realtime:
+
+```sh
+./bin/ai-operator run --env /etc/ai-operator/ai-operator.env --ari-events --call-control --media --voice-provider openai_realtime
+```
+
+## Safety
+
+- OpenAI API key is never printed.
+- Authorization header is never printed.
+- Raw audio is not logged or saved.
+- Base64 audio is not logged.
+- OpenAI is allowed only for explicit commands or `--voice-provider openai_realtime`.
+- Production Kazakhtelecom routes remain disabled.
+
+## Cost Controls
+
+Defaults for test mode:
+
+- max session seconds: 60
+- max input audio bytes: 10MB
+- max output audio bytes: 10MB
+- smoke-test does not call `response.create` unless explicitly requested
+
+## Limitations
+
+- No language/region state machine yet.
+- No RAG/knowledge base yet.
+- No final agent prompt yet.
+- Real tools are parsed but not executed.
+
+## TZ-06 Dialogue Prompt Integration
+
+Voice providers receive state-aware prompt fragments from the Dialogue Orchestrator. The prompt is not trusted as the only guardrail; tool permission is enforced by Go policy before execution.
+
+## TZ-07 Transcript Integration
+
+`VoiceEventUserTranscriptDone` can be passed to the Dialogue Orchestrator for language detection. Assistant transcript and audio events do not trigger language changes.
+
+## TZ-08 Region Resolver Update
+
+TZ-08 adds an offline Kazakhstan Region Resolver with 20 enabled regions, stable `region_code` values, Almaty city/region clarification, false-positive protection, and explicit region-change policy. Production routing remains disabled; OpenAI, PostgreSQL, and Knowledge Base/RAG are still not used at this stage.
+
+## TZ-09 Knowledge Base Update
+
+TZ-09 adds PostgreSQL + pgvector, KGA JSONL ingestion, fake embeddings, hybrid KB search, cross-language fallback for KK queries, citations, and `search_knowledge_base` integration behind the existing language/region guardrail. OpenAI is not called automatically and production routing remains disabled.
+## Tool Result Relay
+
+VoiceProvider ToolCall events are routed through Call Manager to Dialogue Orchestrator. The resulting ToolResult is sent back to the provider with `SendToolResult`. FakeVoiceProvider records sent tool results for offline validation.
+## TZ-11 Handoff Events
+
+VoiceProvider transcript events can trigger handoff detection. ToolCall events for `request_human_handoff` return structured ToolResults to the provider.
+
+
+## Audit Integration
+
+Provider transcript finals and provider errors can be recorded through the audit service. Transcript deltas and raw transcripts are disabled by default.