# Language Selection

## Purpose

Language Selection recognizes an explicit customer choice between Russian (`ru`) and Kazakh (`kk`) before business help is allowed. It is implemented as `internal/dialogue/language` and is independent from Asterisk, OpenAI, and the media pipeline.

## Supported Languages

- `ru`: Russian
- `kk`: Kazakh

The stored language is always one of these strict internal codes.

## Detection Pipeline

1. Normalize text: trim, lowercase, normalize `ё` to `е`, collapse spaces, remove harmless punctuation, preserve Kazakh Cyrillic letters.
2. Match exact codes and phrase tables.
3. Detect mixed RU/KK and ethnicity ambiguity.
4. Apply confidence and reason codes.
5. Run selection policy based on conversation state.

## Russian Phrases

Examples include `ru`, `rus`, `русский`, `русский язык`, `на русском`, `по-русски`, `хочу на русском`, `говорите на русском`, `russian`, `speak russian`, `russki`, and ASR-like variants such as `руский`, `русски`, `порусски`.

## Kazakh Phrases

Examples include `kk`, `kz` as alias to `kk`, `қазақша`, `қазақ тілі`, `қазақ тілінде`, `қазақша сөйлейік`, `казахский`, `на казахском`, `qazaqsha`, `kazakh`, `kazaksha`, and ASR-like variants such as `қазакша`, `казакша`, `показахски`.

## Ambiguous Cases

The detector asks for clarification for cases such as:

- `я русский`
- `я казах`
- `русский или қазақша?`
- `сначала русский потом казахский`

These are not applied as automatic language selection.

## False-Positive Protection

The detector avoids language selection for terms such as `Казахтелеком`, `русский клиент спрашивает`, `У меня вопрос на русском сайте`, embedded `ru`/`kk` substrings, and ordinary business questions.

## Language Change Rules

- In `LANGUAGE_SELECTION`, clear RU/KK choices are applied.
- In `REGION_SELECTION`, language changes require a clear phrase.
- In `READY_TO_HELP` and `QUESTION_ANSWERING`, language changes require explicit switch intent.
- In `HANDOFF`, `CLOSING`, and `ENDED`, language changes are denied.

## Integration

The Dialogue Orchestrator exposes `HandleUserText`. It passes transcripts or CLI input through the detector, applies selection policy, then calls the same `set_language` transition path. `VoiceEventUserTranscriptDone` is wired to this path. Assistant transcript/audio events do not trigger language detection.

## CLI

```bash
./bin/ai-operator doctor --env /etc/ai-operator/ai-operator.env --check-language
./bin/ai-operator language-self-test --env /etc/ai-operator/ai-operator.env
./bin/ai-operator language-self-test --env /etc/ai-operator/ai-operator.env --json
```

## Not Implemented Yet

- Region Resolver
- Knowledge Base/RAG
- final business prompt
- production routing
- real ASR accuracy validation

## TZ-08 Region Resolver Update

TZ-08 adds an offline Kazakhstan Region Resolver with 20 enabled regions, stable `region_code` values, Almaty city/region clarification, false-positive protection, and explicit region-change policy. Production routing remains disabled; OpenAI, PostgreSQL, and Knowledge Base/RAG are still not used at this stage.

## TZ-09 Knowledge Base Update

TZ-09 adds PostgreSQL + pgvector, KGA JSONL ingestion, fake embeddings, hybrid KB search, cross-language fallback for KK queries, citations, and `search_knowledge_base` integration behind the existing language/region guardrail. OpenAI is not called automatically and production routing remains disabled.
## TZ-10 Agent Integration

The agent prompt keeps language selection as the first required action. Business questions in `LANGUAGE_SELECTION` must repeat the language request and must not trigger KB search.