{"absolute_gate_assessment":[{"evidence":"All STT lanes have zero failed requests. Each supplemental arm has one retained raw repetition flag for the same reference-explained, zero-edit phrase; unexplained/added repetitions are zero.","gate":"malformed_empty_timeout_repetition_cases","status":"pass_for_recorded_stt_after_frozen_reference_adjudication","threshold":0},{"evidence":"Recorded STT requests show zero failures; the STT artifacts alone do not independently prove zero service restart/OOM outside request records. Require lifecycle and host telemetry for the full campaign gate.","gate":"crash_oom_parser_loss_unexplained_restart","status":"partially_evidenced","threshold":0},{"gate":"invalid_or_looped_tool_calls","status":"not_applicable_to_stt_and_pending_campaign_evidence","threshold":0},{"gate":"spoken_reasoning_or_tool_markup","status":"not_applicable_to_stt_and_pending_campaign_evidence","threshold":0},{"evidence":"One unique canonical critical honorific/entity case fails in control, Qwen, and Granite across all three repetitions.","gate":"critical_negation_entity_or_number_errors","status":"fail","threshold":0},{"evidence":{"control":0.03064066852367688,"granite":0.033426183844011144,"qwen":0.027855153203342618},"gate":"stt_human_micro_wer_max","status":"pass_all_recorded_arms","threshold":0.1},{"evidence":{"granite_canonical_delta":0.002785515320334262,"granite_supplemental_delta":0.009656652360515022,"qwen_canonical_delta":-0.0027855153203342614,"qwen_supplemental_delta":-0.003218884120171673},"gate":"stt_candidate_wer_max_delta_vs_control","status":"qwen_point_estimate_pass_granite_fail","threshold":0.0},{"evidence":"Canonical sequential, canonical C4, and supplemental artifacts report complete=true for control, Qwen, and Granite with expected human request counts.","gate":"stt_evidence_complete","status":"pass_for_all_nine_recorded_lanes","threshold":true},{"gate":"llm_deterministic_pass_rate","status":"pending_other_campaign_evidence","threshold":1.0},{"gate":"tts_blank_truncated_or_added_critical_speech","status":"pending_other_campaign_evidence","threshold":0},{"gate":"combined_warm_turns_per_arm","status":"unrun","threshold":100},{"gate":"combined_soak_seconds_minimum","status":"unrun","threshold":1200},{"gate":"unexpected_protected_endpoint_loss","status":"unrun","threshold":0},{"evidence":"Observed system swap rose from 33.88 MB to 9,129.94 MB before managed stop (9,096.06 MB observed growth). The evidence does not assign all swap causally to Breeze, but the frozen observational gate is exceeded.","gate":"soak_swap_growth_max_bytes","status":"fail_for_breeze_bf16","threshold":536870912},{"evidence":"First abort observation reported 34% free. This does not cure the independent swap-growth failure.","gate":"abort_system_free_memory_percentage_below","status":"not_triggered_in_breeze_observation","threshold":15},{"gate":"interruption_actual_playback_target_ms","status":"unrun","threshold":250}],"combined_soak_feasibility":{"required_per_arm_evidence":["At least 100 attempted warm conversational turns for candidate and contemporaneous control, with every attempt retained; zero malformed, empty, timeout, crash, parser-loss, unexplained-restart, critical-word, invalid-tool, or spoken-markup failures.","At least 1,200 seconds of representative co-resident STT+LLM+TTS use for each exact arm. If 100 turns complete sooner, continue representative residency/load to 1,200 seconds; if 1,200 seconds elapse sooner, continue through 100 turns.","Matched cold observation kept separate; warm p50/p95 and stage/first-audio timings; report all retries and failures rather than replacing failed attempts.","Real OpenClaw Talk/workbench client coverage: interruptions during speech and reasoning, actual playback stop latency at or below 250 ms, tool-result delays, repeated calls, back-to-back turns, reconnect, cancellation, and recovery.","Before/during/after pressure, free memory, swap, and peak allocation telemetry; swap growth at most 536,870,912 bytes and free memory never below the 15% abort boundary.","Continuous protected-endpoint responsiveness, no unexpected endpoint loss, restart, OOM, audio gap, or unrecorded component substitution; exact revisions/configuration and client identity retained."],"tooling_findings":[{"coverage":"One serialized STT→LLM→TTS replay; explicitly non-realtime and not acoustic playback.","surface":"anvil-serving voice benchmark (voice/benchmark.py)"},{"coverage":"A bounded realtime session with interruption and a completed follow-up; it is not a 100-turn or 20-minute soak runner.","surface":"scripts/voice/realtime_sdk_client_demo.py"},{"coverage":"Can capture interactive real playback/barge-in for a bounded duration; it does not automatically produce matched 100-turn candidate/control evidence or aggregate host/protected-endpoint telemetry.","surface":"scripts/voice/local_loop_demo.py"},{"coverage":"Prove protocol, cancellation, and measurement behavior with fakes; they are not live model, client, audio, or residency evidence.","surface":"unit tests"}],"verdict":"feasible_with_additional_live_orchestration; not established by existing tests or a single built-in command"},"critical_error_gate":{"case_id":"librispeech-test-other-1688-142285-0013","observations":{"control":{"assessment":"critical honorific/entity substitution","reference":"AT LEAST NO FRIEND CAME FORWARDS IMMEDIATELY AND MISSUS THORNTON IS NOT ONE I FANCY TO WAIT TILL TARDY KINDNESS COMES TO FIND HER OUT","repetitions":3,"transcript":"At least no friend came forwards immediately, and mister Thornton is not one, I fancy, to wait till tardy kindness comes to find her out.","word_edits_each":[1,1,1]},"granite":{"assessment":"critical honorific/entity substitution","reference":"AT LEAST NO FRIEND CAME FORWARDS IMMEDIATELY AND MISSUS THORNTON IS NOT ONE I FANCY TO WAIT TILL TARDY KINDNESS COMES TO FIND HER OUT","repetitions":3,"transcript":"at least no friend came forwards immediately and miss thornton is not one i fancy to wait till tardy kindness comes to find her out","word_edits_each":[1,1,1]},"qwen":{"assessment":"critical honorific/entity substitution","reference":"AT LEAST NO FRIEND CAME FORWARDS IMMEDIATELY AND MISSUS THORNTON IS NOT ONE I FANCY TO WAIT TILL TARDY KINDNESS COMES TO FIND HER OUT","repetitions":3,"transcript":"At least, no friend came forwards immediately, and Mister Thornton is not one I fancy to wait till tardy kindness comes to find her out.","word_edits_each":[1,1,1]}},"reason":"Control and Qwen render MISSUS as mister; Granite renders it as miss. Each is wrong under the frozen zero-critical-entity-error gate. A control failure does not relax the gate.","result":"fail_all_arms"},"formatting_interpretation":{"decision":"The numeric renderings are plausible inverse-text-normalization, but the frozen evaluator does not grant ITN equivalence. Preserve the recorded WER; do not silently rescore or waive it.","granite_itn_examples":[{"case_id":"kokoro-agent-02-timer","reference":"Set a timer for twelve minutes and thirty seconds.","reference_words":9,"transcript":"set a timer for 12 minutes and 30 seconds","word_edits":2},{"case_id":"kokoro-agent-03-zip-code","reference":"Find a pharmacy near ZIP code nine eight one zero nine.","reference_words":11,"transcript":"find a pharmacy near zip code 98109","word_edits":5},{"case_id":"kokoro-agent-04-date-time","reference":"Schedule it for Thursday, August fourteenth at three forty-five P.M.","reference_words":12,"transcript":"schedule it for thursday august 14th at 345 p m","word_edits":4},{"case_id":"kokoro-agent-05-extension","reference":"Call the front desk at extension four zero seven.","reference_words":9,"transcript":"call the front desk at extension 407","word_edits":3},{"case_id":"kokoro-agent-06-correction-cancellation","reference":"Change that to seven thirty, no, cancel the reminder.","reference_words":9,"transcript":"change that to 730 no cancel the reminder","word_edits":2}],"normalized_wer":"The frozen evaluator applies Unicode NFKC, case-folding, and punctuation/symbol-to-space normalization. It does not equate number words with digits or ordinals.","raw_cer":"Raw CER compares exact characters, so LibriSpeech uppercase references versus lower/sentence-case hypotheses and punctuation/rendering differences inflate it. Treat it as format-sensitive diagnostic evidence, not semantic equivalence."},"generated_at":"2026-10-06T08:00:01.784278+00:00","human_microphone_coverage":{"evidence":"The supplemental 120-case set is human-recorded LibriSpeech, but the full research plan also requires the user’s voice/accent and realistic microphone conditions. Campaign plan explicitly leaves this separate and unresolved unless supplied or recorded with participation.","promotion_effect":"No STT promotion claim can satisfy the full plan until this coverage is recorded and reviewed.","status":"unresolved_hard_coverage_gap"},"models":{"control":{"identity":{"base_url":"LOCAL_ENDPOINT_REDACTED/v1","language_conditioning":null,"model":"tdt-0.6b-v3","prompt":null},"label":"Parakeet Metal control","lanes":{"c4":{"complete":true,"failed":0,"latency_ms":{"max":289.261167,"p50":173.007375,"p95":279.015006},"micro_wer":0.03064066852367688,"raw_repetition_flags":0,"reference_words":359,"requests":24,"word_edits":11},"canonical":{"complete":true,"failed":0,"latency_ms":{"max":93.407542,"p50":46.616791,"p95":81.681292},"micro_wer":0.03064066852367688,"raw_repetition_flags":0,"reference_words":1077,"requests":72,"word_edits":33},"supplemental":{"complete":true,"failed":0,"latency_ms":{"max":92.756375,"p50":47.514291,"p95":84.591468},"micro_wer":0.034334763948497854,"raw_repetition_flags":1,"reference_words":1864,"requests":120,"word_edits":64}}},"granite":{"identity":{"base_url":"LOCAL_ENDPOINT_REDACTED/v1","checkpoint":"iky1e/granite-speech-5.0-470m-turboctc-mlx-fp16","hardware":"M4 Max 48GiB","language_conditioning":null,"model":"iky1e/granite-speech-5.0-470m-turboctc-mlx-fp16@319abff7072204bc6cd30485aac58da6c1216214","prompt":null,"revision":"319abff7072204bc6cd30485aac58da6c1216214","runtime":"mlx-audio","runtime_version":"0.5.8@70f4add32911bab6f869b824864ad9f1e24dcb97"},"label":"Granite Speech 5.0 470M TurboCTC MLX FP16","lanes":{"c4":{"complete":true,"failed":0,"latency_ms":{"max":114.239041,"p50":71.963416,"p95":109.104431},"micro_wer":0.033426183844011144,"raw_repetition_flags":0,"reference_words":359,"requests":24,"word_edits":12},"canonical":{"complete":true,"failed":0,"latency_ms":{"max":179.364,"p50":24.229625,"p95":40.1615},"micro_wer":0.033426183844011144,"raw_repetition_flags":0,"reference_words":1077,"requests":72,"word_edits":36},"supplemental":{"complete":true,"failed":0,"latency_ms":{"max":42.997584,"p50":21.875605,"p95":37.166627},"micro_wer":0.043991416309012876,"raw_repetition_flags":1,"reference_words":1864,"requests":120,"word_edits":82}}},"qwen":{"identity":{"base_url":"LOCAL_ENDPOINT_REDACTED/v1","checkpoint":"mlx-community/Qwen3-ASR-1.7B-8bit","hardware":"M4 Max 48GiB","language_conditioning":"English","model":"mlx-community/Qwen3-ASR-1.7B-8bit@a8379a2e2f9e313c9292cdf1af4055ab56d50d55","prompt":null,"revision":"a8379a2e2f9e313c9292cdf1af4055ab56d50d55","runtime":"mlx-audio","runtime_version":"0.5.8@70f4add32911bab6f869b824864ad9f1e24dcb97"},"label":"Qwen3-ASR1.7B MLX 8-bit","lanes":{"c4":{"complete":true,"failed":0,"latency_ms":{"max":1235.193792,"p50":564.173104,"p95":1086.949988},"micro_wer":0.027855153203342618,"raw_repetition_flags":0,"reference_words":359,"requests":24,"word_edits":10},"canonical":{"complete":true,"failed":0,"latency_ms":{"max":328.376,"p50":165.368437,"p95":314.251211},"micro_wer":0.027855153203342618,"raw_repetition_flags":0,"reference_words":1077,"requests":72,"word_edits":30},"supplemental":{"complete":true,"failed":0,"latency_ms":{"max":377.425333,"p50":164.676979,"p95":307.583977},"micro_wer":0.03111587982832618,"raw_repetition_flags":1,"reference_words":1864,"requests":120,"word_edits":58}}}},"paired_case_bootstrap":{"granite_independent":[{"candidate_micro_wer":0.033426183844011144,"candidate_minus_control":0.002785515320334262,"candidate_minus_control_ci95":[-0.010169491525423728,0.018461538461538463],"ci_overlaps_zero":true,"cluster_unit":"case_id; all canonical repetitions retained inside each sampled cluster","control_micro_wer":0.03064066852367688,"iterations":20000,"paired_unique_human_cases":24,"quantiles":"nearest-rank empirical 2.5% and 97.5%","rows":[{"candidate_edits":0,"case_id":"librispeech-test-clean-1089-134686-0000","category":"librispeech-test-clean","control_edits":0,"reference_words":84,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-clean-1089-134691-0000","category":"librispeech-test-clean","control_edits":0,"reference_words":15,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-clean-1089-134691-0004","category":"librispeech-test-clean","control_edits":0,"reference_words":27,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-clean-1188-133604-0030","category":"librispeech-test-clean","control_edits":0,"reference_words":12,"repetitions":3},{"candidate_edits":3,"case_id":"librispeech-test-clean-1188-133604-0038","category":"librispeech-test-clean","control_edits":0,"reference_words":57,"repetitions":3},{"candidate_edits":3,"case_id":"librispeech-test-clean-1188-133604-0041","category":"librispeech-test-clean","control_edits":6,"reference_words":81,"repetitions":3},{"candidate_edits":3,"case_id":"librispeech-test-clean-121-123852-0001","category":"librispeech-test-clean","control_edits":3,"reference_words":6,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-clean-121-127105-0001","category":"librispeech-test-clean","control_edits":0,"reference_words":45,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-clean-121-127105-0031","category":"librispeech-test-clean","control_edits":0,"reference_words":57,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-clean-1221-135766-0009","category":"librispeech-test-clean","control_edits":0,"reference_words":99,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-clean-1221-135766-0015","category":"librispeech-test-clean","control_edits":0,"reference_words":24,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-clean-1221-135767-0024","category":"librispeech-test-clean","control_edits":0,"reference_words":51,"repetitions":3},{"candidate_edits":3,"case_id":"librispeech-test-other-1688-142285-0003","category":"librispeech-test-other","control_edits":3,"reference_words":39,"repetitions":3},{"candidate_edits":3,"case_id":"librispeech-test-other-1688-142285-0013","category":"librispeech-test-other","control_edits":3,"reference_words":75,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-other-1688-142285-0047","category":"librispeech-test-other","control_edits":0,"reference_words":12,"repetitions":3},{"candidate_edits":3,"case_id":"librispeech-test-other-1998-29454-0033","category":"librispeech-test-other","control_edits":3,"reference_words":48,"repetitions":3},{"candidate_edits":3,"case_id":"librispeech-test-other-1998-29455-0034","category":"librispeech-test-other","control_edits":3,"reference_words":15,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-other-1998-29455-0038","category":"librispeech-test-other","control_edits":3,"reference_words":102,"repetitions":3},{"candidate_edits":3,"case_id":"librispeech-test-other-2033-164914-0020","category":"librispeech-test-other","control_edits":3,"reference_words":60,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-other-2033-164915-0000","category":"librispeech-test-other","control_edits":0,"reference_words":9,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-other-2033-164916-0007","category":"librispeech-test-other","control_edits":0,"reference_words":42,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-other-2414-128291-0020","category":"librispeech-test-other","control_edits":0,"reference_words":3,"repetitions":3},{"candidate_edits":0,"case_id":"librispeech-test-other-2414-159411-0011","category":"librispeech-test-other","control_edits":0,"reference_words":36,"repetitions":3},{"candidate_edits":12,"case_id":"librispeech-test-other-2414-159411-0014","category":"librispeech-test-other","control_edits":6,"reference_words":78,"repetitions":3}],"scope":"canonical-sequential","seed":20261006,"strata":["librispeech-test-clean","librispeech-test-other"]},{"candidate_micro_wer":0.043991416309012876,"candidate_minus_control":0.009656652360515022,"candidate_minus_control_ci95":[-0.0036344755970924196,0.022601984564498346],"ci_overlaps_zero":true,"cluster_unit":"case_id; all canonical repetitions retained inside each sampled cluster","control_micro_wer":0.034334763948497854,"iterations":20000,"paired_unique_human_cases":120,"quantiles":"nearest-rank empirical 2.5% and 97.5%","rows":[{"candidate_edits":1,"case_id":"librispeech-test-clean-1089-134686-0006","category":"librispeech-test-clean","control_edits":1,"reference_words":24,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1089-134686-0037","category":"librispeech-test-clean","control_edits":0,"reference_words":13,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1089-134691-0003","category":"librispeech-test-clean","control_edits":0,"reference_words":2,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1188-133604-0006","category":"librispeech-test-clean","control_edits":0,"reference_words":8,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-clean-1188-133604-0010","category":"librispeech-test-clean","control_edits":0,"reference_words":15,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1188-133604-0020","category":"librispeech-test-clean","control_edits":0,"reference_words":25,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-121-123859-0003","category":"librispeech-test-clean","control_edits":0,"reference_words":20,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-clean-121-127105-0021","category":"librispeech-test-clean","control_edits":0,"reference_words":5,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-clean-121-127105-0022","category":"librispeech-test-clean","control_edits":0,"reference_words":17,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1221-135766-0008","category":"librispeech-test-clean","control_edits":0,"reference_words":28,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1221-135767-0005","category":"librispeech-test-clean","control_edits":0,"reference_words":14,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1221-135767-0015","category":"librispeech-test-clean","control_edits":0,"reference_words":7,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1284-1180-0016","category":"librispeech-test-clean","control_edits":0,"reference_words":4,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1284-1181-0015","category":"librispeech-test-clean","control_edits":0,"reference_words":17,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1284-134647-0006","category":"librispeech-test-clean","control_edits":0,"reference_words":26,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1320-122617-0015","category":"librispeech-test-clean","control_edits":0,"reference_words":14,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1320-122617-0016","category":"librispeech-test-clean","control_edits":0,"reference_words":31,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1320-122617-0038","category":"librispeech-test-clean","control_edits":0,"reference_words":6,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1580-141083-0029","category":"librispeech-test-clean","control_edits":0,"reference_words":38,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1580-141083-0051","category":"librispeech-test-clean","control_edits":0,"reference_words":6,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1580-141084-0011","category":"librispeech-test-clean","control_edits":0,"reference_words":17,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1995-1826-0001","category":"librispeech-test-clean","control_edits":0,"reference_words":27,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1995-1826-0018","category":"librispeech-test-clean","control_edits":0,"reference_words":16,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-1995-1837-0008","category":"librispeech-test-clean","control_edits":0,"reference_words":6,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-clean-2094-142345-0023","category":"librispeech-test-clean","control_edits":3,"reference_words":21,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-2094-142345-0041","category":"librispeech-test-clean","control_edits":0,"reference_words":1,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-2094-142345-0059","category":"librispeech-test-clean","control_edits":2,"reference_words":30,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-2300-131720-0000","category":"librispeech-test-clean","control_edits":0,"reference_words":13,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-clean-2300-131720-0020","category":"librispeech-test-clean","control_edits":2,"reference_words":30,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-2300-131720-0039","category":"librispeech-test-clean","control_edits":0,"reference_words":4,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-237-134493-0016","category":"librispeech-test-clean","control_edits":0,"reference_words":30,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-clean-237-134500-0025","category":"librispeech-test-clean","control_edits":1,"reference_words":2,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-237-134500-0032","category":"librispeech-test-clean","control_edits":0,"reference_words":15,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-260-123286-0031","category":"librispeech-test-clean","control_edits":0,"reference_words":11,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-260-123288-0007","category":"librispeech-test-clean","control_edits":0,"reference_words":26,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-260-123440-0001","category":"librispeech-test-clean","control_edits":0,"reference_words":2,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-clean-2830-3980-0026","category":"librispeech-test-clean","control_edits":0,"reference_words":2,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-2830-3980-0054","category":"librispeech-test-clean","control_edits":0,"reference_words":29,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-2830-3980-0068","category":"librispeech-test-clean","control_edits":0,"reference_words":12,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-2961-960-0005","category":"librispeech-test-clean","control_edits":0,"reference_words":25,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-2961-961-0005","category":"librispeech-test-clean","control_edits":0,"reference_words":9,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-clean-2961-961-0013","category":"librispeech-test-clean","control_edits":0,"reference_words":10,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-clean-3570-5694-0012","category":"librispeech-test-clean","control_edits":1,"reference_words":12,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-3570-5695-0009","category":"librispeech-test-clean","control_edits":0,"reference_words":14,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-3570-5695-0014","category":"librispeech-test-clean","control_edits":1,"reference_words":33,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-clean-3575-170457-0016","category":"librispeech-test-clean","control_edits":1,"reference_words":2,"repetitions":1},{"candidate_edits":5,"case_id":"librispeech-test-clean-3575-170457-0028","category":"librispeech-test-clean","control_edits":1,"reference_words":9,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-3575-170457-0038","category":"librispeech-test-clean","control_edits":0,"reference_words":31,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-3729-6852-0015","category":"librispeech-test-clean","control_edits":0,"reference_words":4,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-3729-6852-0036","category":"librispeech-test-clean","control_edits":0,"reference_words":12,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-clean-3729-6852-0040","category":"librispeech-test-clean","control_edits":0,"reference_words":26,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-clean-4077-13751-0017","category":"librispeech-test-clean","control_edits":0,"reference_words":16,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-4077-13751-0019","category":"librispeech-test-clean","control_edits":0,"reference_words":8,"repetitions":1},{"candidate_edits":4,"case_id":"librispeech-test-clean-4077-13751-0020","category":"librispeech-test-clean","control_edits":0,"reference_words":26,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-4446-2271-0016","category":"librispeech-test-clean","control_edits":2,"reference_words":31,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-4446-2273-0031","category":"librispeech-test-clean","control_edits":0,"reference_words":4,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-clean-4446-2275-0021","category":"librispeech-test-clean","control_edits":0,"reference_words":14,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-4507-16021-0024","category":"librispeech-test-clean","control_edits":0,"reference_words":7,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-clean-4507-16021-0027","category":"librispeech-test-clean","control_edits":0,"reference_words":3,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-clean-4507-16021-0042","category":"librispeech-test-clean","control_edits":0,"reference_words":17,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-1688-142285-0028","category":"librispeech-test-other","control_edits":0,"reference_words":4,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-1688-142285-0035","category":"librispeech-test-other","control_edits":0,"reference_words":25,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-1688-142285-0049","category":"librispeech-test-other","control_edits":0,"reference_words":12,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-1998-15444-0026","category":"librispeech-test-other","control_edits":1,"reference_words":32,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-1998-29454-0009","category":"librispeech-test-other","control_edits":0,"reference_words":6,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-1998-29454-0018","category":"librispeech-test-other","control_edits":3,"reference_words":14,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-2033-164914-0003","category":"librispeech-test-other","control_edits":0,"reference_words":13,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-2033-164914-0013","category":"librispeech-test-other","control_edits":0,"reference_words":7,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-2033-164915-0016","category":"librispeech-test-other","control_edits":1,"reference_words":26,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-2414-128291-0004","category":"librispeech-test-other","control_edits":0,"reference_words":17,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-2414-128291-0015","category":"librispeech-test-other","control_edits":2,"reference_words":3,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-2414-128292-0031","category":"librispeech-test-other","control_edits":0,"reference_words":8,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-2609-156975-0006","category":"librispeech-test-other","control_edits":1,"reference_words":24,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-2609-156975-0016","category":"librispeech-test-other","control_edits":7,"reference_words":8,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-2609-156975-0024","category":"librispeech-test-other","control_edits":4,"reference_words":5,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-3005-163389-0007","category":"librispeech-test-other","control_edits":0,"reference_words":6,"repetitions":1},{"candidate_edits":7,"case_id":"librispeech-test-other-3005-163391-0010","category":"librispeech-test-other","control_edits":8,"reference_words":26,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-3005-163399-0015","category":"librispeech-test-other","control_edits":1,"reference_words":16,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-3080-5032-0011","category":"librispeech-test-other","control_edits":1,"reference_words":29,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-3080-5040-0016","category":"librispeech-test-other","control_edits":0,"reference_words":19,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3080-5040-0017","category":"librispeech-test-other","control_edits":0,"reference_words":5,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-3331-159605-0009","category":"librispeech-test-other","control_edits":0,"reference_words":25,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3331-159605-0023","category":"librispeech-test-other","control_edits":0,"reference_words":15,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3331-159605-0025","category":"librispeech-test-other","control_edits":0,"reference_words":3,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3528-168669-0068","category":"librispeech-test-other","control_edits":0,"reference_words":41,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3528-168669-0095","category":"librispeech-test-other","control_edits":0,"reference_words":13,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3528-168669-0103","category":"librispeech-test-other","control_edits":0,"reference_words":3,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-3538-142836-0011","category":"librispeech-test-other","control_edits":1,"reference_words":14,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3538-142836-0017","category":"librispeech-test-other","control_edits":0,"reference_words":39,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-3538-142836-0023","category":"librispeech-test-other","control_edits":1,"reference_words":1,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-367-130732-0010","category":"librispeech-test-other","control_edits":2,"reference_words":4,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-367-130732-0011","category":"librispeech-test-other","control_edits":2,"reference_words":19,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-367-293981-0015","category":"librispeech-test-other","control_edits":0,"reference_words":43,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3764-168670-0040","category":"librispeech-test-other","control_edits":0,"reference_words":11,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3764-168671-0001","category":"librispeech-test-other","control_edits":0,"reference_words":24,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3764-168671-0054","category":"librispeech-test-other","control_edits":0,"reference_words":3,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-3997-180297-0004","category":"librispeech-test-other","control_edits":0,"reference_words":13,"repetitions":1},{"candidate_edits":4,"case_id":"librispeech-test-other-3997-182399-0007","category":"librispeech-test-other","control_edits":3,"reference_words":28,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-3997-182399-0014","category":"librispeech-test-other","control_edits":0,"reference_words":6,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-4198-12259-0024","category":"librispeech-test-other","control_edits":1,"reference_words":3,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-4198-61336-0028","category":"librispeech-test-other","control_edits":0,"reference_words":11,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-4198-61336-0030","category":"librispeech-test-other","control_edits":1,"reference_words":19,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-4294-9934-0008","category":"librispeech-test-other","control_edits":0,"reference_words":12,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-4294-9934-0016","category":"librispeech-test-other","control_edits":0,"reference_words":4,"repetitions":1},{"candidate_edits":3,"case_id":"librispeech-test-other-4294-9934-0027","category":"librispeech-test-other","control_edits":2,"reference_words":33,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-4350-10919-0000","category":"librispeech-test-other","control_edits":0,"reference_words":23,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-4350-9170-0022","category":"librispeech-test-other","control_edits":0,"reference_words":5,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-4350-9170-0043","category":"librispeech-test-other","control_edits":0,"reference_words":35,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-4852-28312-0023","category":"librispeech-test-other","control_edits":0,"reference_words":5,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-4852-28312-0031","category":"librispeech-test-other","control_edits":0,"reference_words":12,"repetitions":1},{"candidate_edits":3,"case_id":"librispeech-test-other-4852-28319-0026","category":"librispeech-test-other","control_edits":1,"reference_words":23,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-533-1066-0008","category":"librispeech-test-other","control_edits":0,"reference_words":13,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-533-131556-0005","category":"librispeech-test-other","control_edits":1,"reference_words":23,"repetitions":1},{"candidate_edits":2,"case_id":"librispeech-test-other-533-131564-0022","category":"librispeech-test-other","control_edits":1,"reference_words":3,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-5442-41168-0013","category":"librispeech-test-other","control_edits":1,"reference_words":27,"repetitions":1},{"candidate_edits":1,"case_id":"librispeech-test-other-5442-41169-0002","category":"librispeech-test-other","control_edits":1,"reference_words":5,"repetitions":1},{"candidate_edits":3,"case_id":"librispeech-test-other-5442-41169-0029","category":"librispeech-test-other","control_edits":2,"reference_words":15,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-5484-24317-0031","category":"librispeech-test-other","control_edits":0,"reference_words":31,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-5484-24318-0003","category":"librispeech-test-other","control_edits":0,"reference_words":17,"repetitions":1},{"candidate_edits":0,"case_id":"librispeech-test-other-5484-24318-0010","category":"librispeech-test-other","control_edits":0,"reference_words":8,"repetitions":1}],"scope":"supplemental","seed":20261006,"strata":["librispeech-test-clean","librispeech-test-other"]}],"qwen":[{"bootstrap":"20000 paired case clusters stratified clean/other; seed20261006; canonical3 repeats kept inside case cluster","candidate_micro_wer":0.027855153203342618,"candidate_minus_control_ci95":[-0.017699115044247787,0.01084010840108401],"control_micro_wer":0.03064066852367688,"overlap_zero":true,"paired_unique_human_cases":24,"scope":"canonical-sequential"},{"bootstrap":"20000 paired case clusters stratified clean/other; seed20261006; canonical3 repeats kept inside case cluster","candidate_micro_wer":0.03111587982832618,"candidate_minus_control_ci95":[-0.012249443207126948,0.005696530295183843],"control_micro_wer":0.034334763948497854,"overlap_zero":true,"paired_unique_human_cases":120,"scope":"supplemental"}],"qwen_recorded_source":"qwen-asr-paired-comparison.json"},"promotion_assessment":{"breeze_bf16":"not_eligible_even_if_partial_corpus_later_appears_complete","breeze_reason":"The observed swap-growth gate was exceeded by a wide margin and the run was stopped. Corpus completion cannot waive an absolute resource gate.","granite":"not_eligible","granite_reason":"Point WER is worse than control in canonical and supplemental evidence, violating the non-regression gate; both independent paired intervals overlap zero. Granite also fails the zero-critical-entity gate and user-microphone coverage is missing.","proposed_route":"Retain the current Parakeet Metal STT control. This is a no-winner retention decision, not a claim that the control passes every frozen absolute gate.","qwen":"not_eligible","qwen_reason":"Point WER is lower than control, but both recorded paired 95% intervals overlap zero; the frozen tie rule retains control absent an independently demonstrated critical-word or endpoint benefit. Qwen also fails the zero-critical-entity gate and user-microphone coverage is missing."},"repetition_adjudication":{"adjudicated_unexplained_or_added_repetitions_per_arm":{"control":0,"granite":0,"qwen":0},"basis":"Reference contains “up and down up and down”; all transcripts preserve it with zero word edits. Raw artifacts remain unchanged.","case_id":"librispeech-test-clean-237-134500-0032","raw_supplemental_flags_per_arm":{"control":1,"granite":1,"qwen":1}},"review_scope":"Read-only independent review of frozen plan and recorded STT evidence; no host operation or candidate inference.","schema":"anvil-serving.independent-stt-assessment/v1","source_sha256":{"baseline-repetition-adjudication.json":"bf6d58c4ff097ae8ecab49d1720f763f93a00a26aaacf971e58bf81a063cb142","baseline-stt-canonical-concurrent.json":"986087faa7e15449276975a298499828f604587cc598c80ae9a3e399dc1c2737","baseline-stt-canonical-sequential.json":"f1ce209012747ab17c1f358484fd74738347cfbc3b0bca6a555632df53995c73","baseline-stt-supplemental.json":"f05e9f3a6adc663b809ec377534b395d228e09a72ddb54893a0b7f8e983c6d6f","breeze-bf16-abort-assessment.json":"ce9cb43c8aa17d24040071f180e2fc245fe449e842ac3a90bfb7888c01aa51fa","campaign-compatibility-amendment.json":"0718219787ef5c1a3a6da1798a2d52e99afbf362fe411394c9f8edf0778924f2","campaign-plan.json":"bfbebed576ac37d62def3e61298b247696a7c6b9aecc979057c91e80bfb623b4","granite-fp16-canonical-concurrent.json":"6c1749e5045a7b616d0903246f46b495fb7b41245e72116506d56aa873684016","granite-fp16-canonical-sequential.json":"5af8946539f7a83823b1b51be34acdaa1f6223a8dc1c8ae9241b77009f46e274","granite-fp16-supplemental.json":"bb610c05d54aaf1675ac502c9bb9947094899d1fe724a321cce98ba840736f8a","public:anvil_serving/voice/benchmark.py":"072a1b2d45a5448260a77b32614b3a2b5071a87fe47116bd86e06193c60bb3c8","public:anvil_serving/voice/stt_benchmark.py":"45ec35b02345dc2a65fdc9c5c44500719fee29dc06241146feb1df20d8f9b10b","public:scripts/voice/local_loop_demo.py":"4b7d2d151ef68a72354dc955daca1686b27d73a3eb911f9578411a8d4d9b7798","public:scripts/voice/realtime_sdk_client_demo.py":"760366c7c82a268ab5ad2927737e44a51efd415a6f575ff148a3d4364b40d5c1","qwen-asr-canonical-concurrent.json":"d90e7899dbbf0770b27b9a22af2651356465b3f830cdf1c884f72b278dfed95e","qwen-asr-canonical-sequential.json":"d1995500f80f1de91d5b8d7a1bdd3a6daa9b0fd426099b0365c37a5f46317592","qwen-asr-paired-comparison.json":"5938872bf622782f6784ff9a0b6ec14b64b307306ba8cd2613566808a6ec2a2f","qwen-asr-supplemental.json":"2a2ee12d0ab24908c718bae5b45c06a44f4cd0a763015f9c45bb071cbdbbf177"}}