Play a filler ack during slow voice decisions and cache KB search rows #2
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: digiops/call-center#2
Reference in New Issue
Block a user
Two independent latency fixes for the voice-assistant reply pipeline,
both scoped to the parts of the flow that run regardless of whether
voice_v2 is enabled for a queue:
media_runtime._process_utterance: the v1/fallback turn path (used by
any queue not covered by AI_VOICE_V2_QUEUE_CODES) silently awaited the
full LLM decision with no audio playing at all, unlike the v2 path
which already has a decision-timeout ack. Give v1 the same behavior:
wait up to 600ms (_v1_ack_wait_seconds) for the decision, and if it's
still not ready, play a short "Секунду." filler via the existing
_emit_early_ack before the real reply, instead of leaving the caller
in silence for the full LLM+TTS round trip. Reuses the same ack
selection/playback code path v2 already exercises, so no new failure
modes - just an added timeout branch mirroring the existing v2 one.
ai_orchestrator_service._kb_search: every voice/chat turn re-ran a
full-table scan of kb_articles (all columns, including body text) and
rescored every row in Python, even though the KB rarely changes
mid-conversation. Added an in-process cache keyed by language, gated
on a cheap content fingerprint (row count + max id + max updated_at +
summed title/body/tags length, all computed server-side without
transferring the text columns). A fingerprint mismatch always
triggers a fresh fetch, so this can never serve stale results after
an insert/update/delete - unlike a naive TTL cache, which would have
been be wrong the moment a test (or a real KB edit) changed the table
within the cache window.
Note the first fingerprint design (count + max id + max updated_at
only) was insufficient: utc_now_iso() truncates to whole seconds and
SQLite reuses primary keys after a full-table delete, so two
different row sets written in the same wall-clock second could share
a fingerprint. Caught this via a real test failure
(test_ai_whatsapp_relaxed_kb_search_answers_phrase_query breaking
only when run after test_ai_orchestrator_service.py in the same
process) before it could reach production; the summed content-length
term closes the gap.
Added test_media_runtime_plays_filler_ack_when_v1_decision_is_slow
(asserts greeting -> ack -> reply delivery order when process_turn is
slow) and verified the KB cache against the full
test_ai_orchestrator_service.py + test_ai_whatsapp_orchestrator_service.py
suite plus a wider kb/orchestrator/whatsapp/telegram/voice-filtered run:
only the same pre-existing, already-documented failures remain (unrelated
sales_service test-isolation ordering, one known persona-prompt
assertion) - no new failures from either change.
Streaming the LLM decision itself (start speaking reply_text before the
full structured JSON response finishes generating) was scoped but
deliberately deferred: it needs incremental JSON parsing on top of SSE
streaming to detect when just the reply_text field is complete, shared
across both voice and text-channel decision paths - a separate,
higher-risk change that deserves its own PR and testing pass rather than
being bundled here.
Ревью (Arystanbek):
Чистая реализация, можно мержить:
asyncio.shield(decision_task)— правильно, таймаут не отменяет таск решения, потом досылается реальный ответ.actor.early_ack_startedзащищает от двойного ack. OKcount+max_id+max_updated_at+ сумма длинtitle/body/tags_json, всё server-side черезfunc.length) — это не TTL: при любом insert/update/delete fingerprint меняется -> рефетчит заново, стейлых данных не отдаёт.session.expungeкорректно отцепляет ORM-объекты для переиспользования. OKgreeting-> "Секунду." ->replyпри slowprocess_turn. OKCI/CD:
media_runtime.py-> контейнерai-voice-runtime-service.konturai-deploy.sh(casecall-center) после мержа перезапустит толькоapi-gateway—voice-runtimeне обновится автоматически, нужен ручный рестарт этого сервиса, иначе filler-ack не встанет в проде.