Initial import with GitLab CI/CD and registry deploy flow
This commit is contained in:
@@ -0,0 +1,59 @@
|
||||
# UAT Pilot Package
|
||||
|
||||
This folder contains the operational UAT package for Stage 4 pilot acceptance.
|
||||
|
||||
## Scope Control
|
||||
|
||||
- The frozen MVP pilot scope is defined in `docs/gates/mvp-pilot-baseline.md`.
|
||||
- Out-of-scope requests must be redirected to `docs/roadmap/05-wave2-backlog.md`.
|
||||
- During pilot close-out, only `P1` and `P2` defects are remediated in MVP.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Run preflight before participants join:
|
||||
- Existing environment: `python scripts/uat_preflight.py --base-url http://<gateway-host>:8080`
|
||||
- Local auto-start: `python scripts/uat_preflight.py --auto-start`
|
||||
2. Run automated dry-run and generate UAT package:
|
||||
- Existing environment: `python scripts/uat_dry_run.py --base-url http://<gateway-host>:8080 --update-defect-register`
|
||||
- Local auto-start: `python scripts/uat_dry_run.py --auto-start --update-defect-register`
|
||||
3. Create/adjust a session protocol from template (or from generated dry-run package):
|
||||
- `docs/uat/session-template.md`
|
||||
- Or prepare a full manual bundle:
|
||||
- `python scripts/uat_manual_prepare.py --environment-url http://<pilot-gateway>:8080`
|
||||
4. Execute all scenarios:
|
||||
- `docs/uat/scenario-checklist.md`
|
||||
5. Register all defects:
|
||||
- `docs/uat/defect-log-template.csv`
|
||||
6. Capture sign-off:
|
||||
- `docs/uat/signoff-template.md`
|
||||
7. Attach preflight report from:
|
||||
- `docs/uat/evidence/`
|
||||
8. After signatures and zero open `P1/P2`, close Gate 4:
|
||||
- Optional validation only:
|
||||
- `python scripts/finalize_mvp_pilot.py --session-dir docs/uat/evidence/manual_<session-id> --dry-run`
|
||||
- `python scripts/finalize_mvp_pilot.py --session-dir docs/uat/evidence/manual_<session-id>`
|
||||
|
||||
## Triage Rules
|
||||
|
||||
- `P1`: pilot acceptance is impossible
|
||||
- `P2`: key MVP user journey is broken
|
||||
- `P3`: acceptable with limitation, defer unless it blocks sign-off
|
||||
- `P4`: cosmetic or low-priority backlog item
|
||||
|
||||
`P3` and `P4` findings should be logged and moved to post-MVP backlog instead of
|
||||
stretching the MVP remediation cycle.
|
||||
|
||||
## Mandatory Evidence Bundle
|
||||
|
||||
- preflight report
|
||||
- completed session protocol
|
||||
- completed scenario checklist
|
||||
- filled defect log
|
||||
- signed sign-off sheet
|
||||
- updated `docs/gates/p1-p2-defects.md`
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
- All mandatory scenarios executed
|
||||
- No open P1/P2 defects
|
||||
- Signed protocol by business and IT owners
|
||||
@@ -0,0 +1,2 @@
|
||||
defect_id,severity,status,scenario,service,risk_level,summary,repro_steps,actual_result,expected_result,verification_step,owner,opened_at,closed_at,backlog_bucket,notes
|
||||
UAT-001,P3,Open,S4,routing-service,Medium,"Example defect summary","1) ... 2) ...","Actual behavior","Expected behavior","Re-run scenario S4 after fix",owner_name,2026-02-26T00:00:00Z,,Wave2,
|
||||
|
@@ -0,0 +1,3 @@
|
||||
dry_run_*/
|
||||
manual_*/
|
||||
preflight_*.md
|
||||
@@ -0,0 +1,8 @@
|
||||
# UAT Evidence
|
||||
|
||||
Store generated pilot artifacts in this folder, for example:
|
||||
|
||||
- Preflight reports from `python scripts/uat_preflight.py`
|
||||
- Session protocol copies
|
||||
- Signed acceptance documents
|
||||
- Exported logs/screenshots
|
||||
@@ -0,0 +1,223 @@
|
||||
# UAT Scenario Checklist
|
||||
|
||||
## S0 Login and RBAC deny
|
||||
|
||||
- Preconditions:
|
||||
- Test users are available (`admin`, `supervisor`, `operator`, `analyst`)
|
||||
- Steps:
|
||||
- Sign in through the gateway
|
||||
- Verify role-dependent access to main flows
|
||||
- Attempt a denied privileged action as `operator` (for example create a user)
|
||||
- Expected:
|
||||
- Login succeeds
|
||||
- Allowed actions work for the selected role
|
||||
- Denied privileged action returns `403`
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- user:
|
||||
- screenshot/log:
|
||||
|
||||
## S1 Customer create and search
|
||||
|
||||
- Preconditions:
|
||||
- Operator UI or API access is available
|
||||
- Steps:
|
||||
- Create a customer
|
||||
- Search the customer by name or phone
|
||||
- Open the customer card
|
||||
- Expected:
|
||||
- Customer is created once
|
||||
- Search returns the same customer
|
||||
- Customer card data is consistent
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- customer_id:
|
||||
- screenshot/log:
|
||||
|
||||
## S2 Voice interaction lifecycle
|
||||
|
||||
- Preconditions:
|
||||
- Operator account is available
|
||||
- Voice queue is available
|
||||
- Steps:
|
||||
- Create a voice interaction
|
||||
- Assign interaction to operator
|
||||
- Change status to `closed`
|
||||
- Open the interaction timeline
|
||||
- Expected:
|
||||
- Status transitions are correct
|
||||
- Timeline contains creation, assignment, and close events
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- interaction_id:
|
||||
- screenshot/log:
|
||||
|
||||
## S3 Assignment and escalation to second line
|
||||
|
||||
- Preconditions:
|
||||
- Queue routing rule for escalation exists
|
||||
- Steps:
|
||||
- Create or select an active interaction
|
||||
- Assign it to an operator
|
||||
- Escalate it to the target queue
|
||||
- Expected:
|
||||
- Assignment is accepted
|
||||
- Queue is changed to the target queue
|
||||
- Status changes to `escalated`
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- interaction_id:
|
||||
- target_queue:
|
||||
|
||||
## S4 Routing and timeline verification
|
||||
|
||||
- Preconditions:
|
||||
- Queue rules are configured
|
||||
- At least one completed interaction exists
|
||||
- Steps:
|
||||
- Execute a route request for the active queue
|
||||
- Verify assigned operator and SLA result
|
||||
- Re-open interaction timeline
|
||||
- Expected:
|
||||
- Routing returns assignee and SLA
|
||||
- Timeline remains complete and chronological
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- queue_id:
|
||||
- route output:
|
||||
|
||||
## S5 Voice event intake
|
||||
|
||||
- Preconditions:
|
||||
- Voice adapter endpoint is reachable
|
||||
- Steps:
|
||||
- Submit a voice event (`call.started`, `ivr.completed`, or `call.ended`)
|
||||
- Verify event is stored and retrievable
|
||||
- Expected:
|
||||
- Voice event is accepted
|
||||
- Event ID is returned
|
||||
- Event can be listed afterwards
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- event_id:
|
||||
- call_id:
|
||||
|
||||
## S6 Telegram interaction lifecycle
|
||||
|
||||
- Preconditions:
|
||||
- Telegram adapter endpoint is reachable
|
||||
- Steps:
|
||||
- Send webhook payload to Telegram adapter
|
||||
- Verify message registration
|
||||
- Verify interaction visibility in operator workflow
|
||||
- Expected:
|
||||
- Message is accepted
|
||||
- Related interaction or adapter state is traceable
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- message_id:
|
||||
- screenshot/log:
|
||||
|
||||
## S7 KB usage in active handling
|
||||
|
||||
- Preconditions:
|
||||
- KB category/article created by analyst/supervisor
|
||||
- Steps:
|
||||
- Search KB by keyword
|
||||
- Open article
|
||||
- Use article during interaction processing
|
||||
- Expected:
|
||||
- Relevant article found
|
||||
- Operator confirms article usability
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- article_id:
|
||||
- keyword:
|
||||
|
||||
## S8 Supervisor realtime
|
||||
|
||||
- Preconditions:
|
||||
- At least two agent states are submitted
|
||||
- Steps:
|
||||
- Set agent states (`READY`, `BUSY`)
|
||||
- Update queue metrics
|
||||
- Open realtime view/API
|
||||
- Expected:
|
||||
- Agent totals and states are visible
|
||||
- Queue snapshot is visible
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- timestamp:
|
||||
- screenshot/log:
|
||||
|
||||
## S9 KPI report and export validation
|
||||
|
||||
- Preconditions:
|
||||
- KPI events are ingested for the test queue
|
||||
- Steps:
|
||||
- Open `/reports/kpi`
|
||||
- Verify KPI payload
|
||||
- Open `/reports/export` and capture CSV output
|
||||
- Expected:
|
||||
- KPI payload contains `SL`, `ASA`, `AHT`, `Abandon`, `FCR`
|
||||
- Export contains the expected header and queue rows
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- queue_id:
|
||||
- export snippet:
|
||||
|
||||
## O1 Backup and restore verification
|
||||
|
||||
- Preconditions:
|
||||
- Pilot environment contains test data
|
||||
- Steps:
|
||||
- Run backup script
|
||||
- Restore into a clean target
|
||||
- Verify the restored data set
|
||||
- Expected:
|
||||
- Archive is created
|
||||
- Restore succeeds
|
||||
- Core interaction data remains intact
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- archive:
|
||||
- restore check:
|
||||
|
||||
## O2 Load test baseline (`100 voice + 100 digital`)
|
||||
|
||||
- Preconditions:
|
||||
- Pilot environment is stable
|
||||
- Steps:
|
||||
- Run `python scripts/load_test.py --base-url http://<gateway-host>:8080 --voice 100 --digital 100`
|
||||
- Record success and latency output
|
||||
- Expected:
|
||||
- Load completes without blocking errors
|
||||
- MVP baseline target is met
|
||||
- Result:
|
||||
- [ ] Pass
|
||||
- [ ] Fail
|
||||
- Evidence:
|
||||
- command:
|
||||
- summary:
|
||||
@@ -0,0 +1,72 @@
|
||||
# UAT Session Protocol
|
||||
|
||||
## Session Metadata
|
||||
|
||||
- Session ID:
|
||||
- Date:
|
||||
- Start time:
|
||||
- End time:
|
||||
- Environment URL:
|
||||
- Build/version:
|
||||
- Deployment date:
|
||||
- Cluster/namespace:
|
||||
|
||||
## Scope Confirmation
|
||||
|
||||
- [ ] MVP pilot scope reviewed against `docs/gates/mvp-pilot-baseline.md`
|
||||
- [ ] Out-of-scope requests redirected to `docs/roadmap/05-wave2-backlog.md`
|
||||
- [ ] API stabilization rules acknowledged (backward-compatible fixes only)
|
||||
|
||||
## Participants
|
||||
|
||||
- Operators:
|
||||
- Supervisor:
|
||||
- Analyst:
|
||||
- Admin:
|
||||
- Business owner:
|
||||
- IT owner:
|
||||
|
||||
## Preconditions
|
||||
|
||||
- [ ] Preflight report attached (`docs/uat/evidence/preflight_*.md`)
|
||||
- [ ] Test accounts are active
|
||||
- [ ] Test queue is configured
|
||||
- [ ] Recording and audit logging are enabled
|
||||
- [ ] Defect log prepared from `docs/uat/defect-log-template.csv`
|
||||
|
||||
## Execution Summary
|
||||
|
||||
- Mandatory scenarios executed:
|
||||
- Passed:
|
||||
- Failed:
|
||||
- Blocked:
|
||||
|
||||
## Defect Summary
|
||||
|
||||
- P1:
|
||||
- P2:
|
||||
- P3:
|
||||
- P4:
|
||||
- Defect log attachment:
|
||||
|
||||
## Decision
|
||||
|
||||
- [ ] Accepted with conditions
|
||||
- [ ] Accepted for pilot completion
|
||||
- [ ] Not accepted
|
||||
|
||||
Accepted with conditions is allowed only when all `P1` and `P2` items are closed and
|
||||
remaining `P3`/`P4` findings are explicitly deferred to `Wave 2`.
|
||||
|
||||
## Comments
|
||||
|
||||
- Business comments:
|
||||
- IT comments:
|
||||
- Deferred backlog references:
|
||||
|
||||
## Signatures
|
||||
|
||||
- Business owner:
|
||||
- IT owner:
|
||||
- Supervisor representative:
|
||||
- Date:
|
||||
@@ -0,0 +1,48 @@
|
||||
# UAT Sign-off Sheet
|
||||
|
||||
## Scope Confirmation
|
||||
|
||||
- MVP pilot scope confirmed against `docs/gates/mvp-pilot-baseline.md`
|
||||
- Out-of-scope items acknowledged and deferred to `docs/roadmap/05-wave2-backlog.md`
|
||||
- No new feature requests are included as MVP defects
|
||||
|
||||
## Acceptance Statement
|
||||
|
||||
The undersigned confirm that UAT scenarios were executed for MVP pilot scope and
|
||||
results are recorded in the session protocol.
|
||||
|
||||
## Conditions
|
||||
|
||||
- No open P1 defects
|
||||
- No open P2 defects
|
||||
- Mandatory scenarios completed
|
||||
|
||||
## Known Exclusions
|
||||
|
||||
- LDAP / SSO
|
||||
- additional channels beyond `voice` and `Telegram`
|
||||
- standalone supervisor/admin UI
|
||||
- advanced IVR builder
|
||||
- full recording suite
|
||||
- extended KPI catalog
|
||||
|
||||
## Decision
|
||||
|
||||
- [ ] Accepted for pilot completion
|
||||
- [ ] Accepted with conditions (only `P3`/`P4` deferred)
|
||||
- [ ] Not accepted
|
||||
|
||||
## Signatures
|
||||
|
||||
- Business owner:
|
||||
- Name:
|
||||
- Signature:
|
||||
- Date:
|
||||
- IT owner:
|
||||
- Name:
|
||||
- Signature:
|
||||
- Date:
|
||||
- Supervisor representative:
|
||||
- Name:
|
||||
- Signature:
|
||||
- Date:
|
||||
Reference in New Issue
Block a user