Final request-to-evidence coverage¶
This was a bounded quality/context scout, not a controlled serving-performance comparison. The campaign closed with no-promotion. Exact evaluation routes/output caps and the GLM incumbent were restored; the router runs a reviewed PR #704 runtime overlay rather than the original router image.
| Requested outcome | Required gate | Retained evidence | Status | Claim allowed | Remaining limit |
|---|---|---|---|---|---|
| Functional agentic behavior | Six-case suite per recipe | Three native agentic reports | covered | Each recipe passed 6/6 | No broader quality inference |
| Coding quality | Six Aider cases and five attempted SWE tasks per recipe | Three Aider report pairs; three SWE results and official grader summaries | partial | Preserve fixed denominators and graded/ungraded outcomes | Historical Aider attribution is uncertain; SWE results remain as officially graded |
| Context retrieval | Nine samples at requested 8,192 / 32,768 / 114,688 buckets | Three native context results | covered | 9/9 per recipe through maximum actual prompt 114,588; Flare effective context 114,688 | Full 262,144 advertised/configured context was not tested |
| Serving output | Bounded C1 strict-output cells | vLLM and Flare native C1 results; Flare reconciliation | failed output gates | Both C1 cells received eight responses and had 0/8 performance-eligible; C4/C8 not run | No speed/capacity result |
| Candidate restoration | Restore exact prior routes/caps and incumbent; post-run direct/routed checks | Final restoration evidence and sanitized preflights | passed with explicit runtime-image difference | Exact route/cap config and GLM incumbent restored; local direct/routed 5/5 and one Pi read-tool turn passed | Router uses reviewed PR #704 overlay, not original image; no whole-fleet, reboot, or fresh-host claim |
| Promotion | All required qualification and integration gates | Final no-promotion decision summary | no-promotion | No primary promotion or candidate route assignment | Mixed quality, failed C1 strict output, and missing workload/run-plan artifacts remain qualification limits |
Serving-comparison metric matrix¶
No controlled speed-comparison matrix was declared. TTFT, decode, TPOT, E2E, throughput, performance p50/p95, and concurrent capacity were not collected as qualification evidence. Configured context and concurrency remain distinct from actual prompt lengths and measured C1 workload.