OpenClaw Talk voice latency candidate A/B status report (2026-07-08)¶
Status: synthesis of the
voice-latency-model-abevidence set. This is not completed candidate A/B evidence: the baseline was measured, but the candidate rows failed before reaching the LLM stage. This report does not promote a model, change router policy, change[router].profile_path, or alter OpenClaw production model selection. Historical note: a later Fast-tier promotion changed the production Fast route toqwen36-35b-a3b-nvfp4; see2026-07-08-fast-tier-promotion.md.
Executive Determination¶
Do not promote a new voice LLM candidate yet.
Candidate model A/B testing was incomplete at the time of this report. The successful measurement was the then-current baseline only. Do not treat the retained candidate failure rows as proof that the candidate models were tested.
Topology correction after follow-up review: mini-audio is a supported
optional Mini-local audio mode, but it is not the intended topology for the
candidate benchmark workflow. Companion Node's 16 GB RAM is reserved for OpenClaw
Gateway, Anvil Voice Realtime/proxy, Claude Code, and Codex. Do not run
STT/TTS/LLM model serves on Mini for reference Talk validation or candidate
A/B. The intended candidate benchmark needs remote or proxied audio, such as
dark-audio or mini-dark-audio-proxy, after the matching bridge/proxy is
verified.
The current OpenClaw Talk path is functionally healthy: spoken turns reach the main chat session, session context is retained, tool calls work, hidden control text is absent from visible user prompts, and the duplicate-message spam did not recur in the checked session.
The latency evidence is not yet a valid candidate-model A/B. The only
successful timing row is the Mini-run baseline for baseline-qwen36-27b on the
mini-audio profile. The candidate rows were retained as evidence, but they
failed before STT because they were executed from a non-gateway checkout whose
127.0.0.1 was not Companion Node's loopback. Those failures prove the topology
guardrail, not candidate model speed or quality.
Original recommendation: keep the then-current production fast Talk model in place and
rerun the intended-topology baseline and actual candidate tests from Fakoli
Mini with the correct audio profile plus --candidate-overlay. Only reconsider
promotion after comparable candidate timing and live Talk validation both pass.
What Was Tested¶
The evidence set covered these phases:
| Phase | Evidence | Result |
|---|---|---|
| Baseline | mini-audio using baseline-qwen36-27b through Primary Node router |
Successful supported-mode timing row captured; not the intended remote-audio benchmark baseline |
| Shortlist | Qwen3 dense NVFP4 and Gemma 4 dense candidates | Candidate set defined; no promotion implied |
| Matrix | Baseline plus candidate profiles and tool-relevant weather turn | Baseline measured; candidate rows failed before STT from wrong host context, so candidate models were not benchmarked |
| Live Talk validation | Companion Node gateway and voice runtime with Dark router | Functional Talk path passed with warning-only COLO smoke |
| Final recommendation | Promotion readiness synthesis | Needs more comparable Mini-run data before model change |
Current Reference Topology¶
The interpretation depends on where a command is executed:
| Host | Owns |
|---|---|
| Companion Node | OpenClaw Gateway and Anvil Voice Realtime/proxy only in the reference path. Its 16 GB RAM is reserved for OpenClaw, Claude Code, and Codex. |
| Primary Node | Anvil router at http://100.64.0.10:8000/v1, candidate LLM serves, and the intended STT/TTS model hosting or bridge boundary for candidate testing. |
| Mini proxy profile | mini-dark-audio-proxy loopback endpoints on Mini at 127.0.0.1:30110 and 127.0.0.1:30111, only valid after Mini-side proxy listeners are up |
| Optional Mini-local profile | mini-audio, valid only for explicit same-host/local-audio validation, not normal candidate A/B |
A Windows/operator checkout cannot validate Mini-local audio by calling its own
127.0.0.1. Candidate runs that fail on those loopback ports from a non-gateway
host are topology negative controls.
Timing Results¶
The successful measured timing row is the rerun mini-audio baseline. Treat it
as a supported-mode measurement, not as the intended remote-audio candidate
benchmark baseline:
| Profile | Candidate | Status | TTFA ms | Turn ms | STT ms | LLM ms | TTS ms |
|---|---|---|---|---|---|---|---|
mini-audio |
baseline-qwen36-27b |
measured prior rerun | 611.29 | 789.06 | 106.28 | 356.82 | 325.95 |
Stage share of total turn latency:
| Stage | Time ms | Share |
|---|---|---|
| STT | 106.28 | 13.5% |
| LLM | 356.82 | 45.2% |
| TTS | 325.95 | 41.3% |
Determination from this row: the LLM and TTS stages are co-dominant for the
mini-audio mode. This does not establish the latency split for the intended
remote/proxied-audio candidate benchmark topology.
Candidate Matrix Outcome¶
The candidate rows are retained in
tests/fixtures/operator_workflows/voice_latency_model_ab_matrix.json, but none
of them produced a valid latency comparison or a candidate model timing:
| Candidate row | Status | Determination |
|---|---|---|
candidate-qwen3-32b generated-audio run |
Failed before STT | Invalid latency comparison; wrong loopback context |
candidate-gemma4-12b generated-audio run |
Failed before STT | Invalid latency comparison; wrong loopback context |
candidate-gemma4-e4b generated-audio run |
Failed before STT | Invalid latency comparison; wrong loopback context |
candidate-qwen3-32b weather/location tool turn |
Blocked before live tool validation | Cannot evaluate tool-call regression yet |
The candidate failures should not be interpreted as model failures, and they should not be counted as completed model tests. They show that LLM-only candidate overlays must be run with a verified Dark-host or Mini-proxied audio profile, so that the STT/TTS path is valid before the candidate LLM is measured.
Functional Talk Result¶
Live OpenClaw Talk validation on the Mini-to-Dark path passed the behaviors that were causing concern earlier:
| Behavior | Result |
|---|---|
| Session transcript delivery | Visible spoken turns reached the active main session |
| Session memory | Conversation context persisted across turns |
| Tool calls | Weather request produced an exec tool call and weather result |
| Hidden forced-consult text | Not present in the checked visible session history |
| Duplicate message spam | No repeated dense burst recurred after cleanup in the checked window |
| COLO smoke | Exited 0; verdict was warn because the command did not include --run-generations |
The live COLO interaction benchmark recorded 10 interaction benchmark requests,
10 HTTP 200 statuses, and finish reason stop for all 10. Latency p50/p95
was 568.6 ms / 1259.9 ms, and exact-generation throughput p50/p95 was
82.77 / 171.82 tokens/sec.
Pass / Fail¶
| Gate | Verdict | Reason |
|---|---|---|
| OpenClaw Talk functional path | Pass | Memory, tools, transcript delivery, and cleanup checks passed |
| Hidden prompt pollution fix | Pass | Checked session contained zero visible forced-consult/control-text markers |
| Duplicate spam fix | Pass with monitoring | Historical burst remains in old logs, but no new sustained repeat sequence was observed |
| Baseline latency capture | Pass | Mini-run baseline captured with TTFA 611.29 ms and turn latency 789.06 ms |
| Candidate latency A/B | Incomplete / fail for promotion evidence | Candidate rows did not reach the LLM stage, so candidate models were not actually tested |
| Model promotion readiness | Fail / not ready | No successful candidate timing plus live Talk regression check exists |
| Cost-control gate | Pass | No cloud model path or metered promotion was introduced |
Historical Decision¶
At the time of this report, keep the then-current baseline-qwen36-27b fast
Talk path in production.
Do not promote Qwen3-32B, Gemma 4 12B, or Gemma 4 E4B from the current evidence. They remain untested candidates, not rejected models. The evidence is insufficient because the candidate runs did not exercise the full voice path from the correct host/topology and did not reach candidate LLM timing.
Latency work should continue on two tracks:
- Re-run the baseline and model candidates with Companion Node as
gateway/realtime/proxy only, using the intended audio profile, likely
dark-audioormini-dark-audio-proxy, after verifying the bridge/proxy endpoints. - Keep audio topology fixed and change only the LLM via
--candidate-overlay. - Investigate TTS latency/chunking alongside LLM latency, because the supported-mode baseline shows TTS can be nearly as large as the LLM stage.
Required Next Evidence Before Promotion¶
Before any production model or routing profile change, capture:
| Required evidence | Acceptance condition |
|---|---|
| Fresh intended-topology baseline JSON | Same command host, audio profile, prompt shape, and evidence output path as candidate runs |
| At least one successful candidate overlay JSON | Candidate differs only by LLM overlay; STT/TTS profile remains comparable |
| Tool-relevant live Talk turn | Weather/location or equivalent tool call succeeds after candidate selection |
| Session transcript check | Spoken user and assistant turns land in the main session without hidden control text |
| Duplicate-message scan | No sustained repeat burst after candidate live turn |
| Human promotion gate | router_promote or anvil-serving router promote remains explicitly human-approved |
Recommended first candidate order remains:
nvidia/Qwen3-32B-NVFP4as the closest dense Qwen operational comparison.google/gemma-4-12B-itas the main Gemma dense scout.google/gemma-4-E4B-itonly to measure the fastest plausible lower-quality bound for spoken responsiveness.
Source Evidence¶
docs/findings/2026-07-07-voice-latency-baseline.mddocs/findings/2026-07-07-voice-latency-model-shortlist.mddocs/findings/2026-07-08-voice-latency-candidate-matrix.mddocs/findings/2026-07-08-openclaw-talk-live-validation.mddocs/findings/2026-07-08-voice-latency-final-recommendation.mdtests/fixtures/operator_workflows/voice_latency_model_ab_matrix.json