Benchmarks¶
2026-11-03 RTX5090 research update: the candidate campaign records bounded coding and synthetic capability results without changing any deployment decision. The Ignis note separates model, runtime and evaluator improvements. Dossier.
Current review, 2026-11-01: Swift TP1 has qualification accepted on one RTX PRO 6000 Blackwell Max-Q. This is a qualification result; the measured C2/C1 tradeoff, strict-format failure, and SWE limit are retained in the dated finding.
Separate private-target scouts, 2026-11-01: original configuration retained. The cache-layout and DFlash scouts each completed 144 timed requests on the dual RTX PRO 6000 pair. Neither candidate improved median response latency by 10% against both controls. Cache slots increased 38.60% with the layout change and decreased 40.63% with DFlash2 K3. Both candidates were removed and the original configuration was restored. That scout target identity is private; the named October reference below is historical evidence, not a statement of today's selected checkpoint. Private-target dossier.
Historical primary recommendation, 2026-10-17: the operator selected
nvidia/GLM-5.3-Flash-FP4-Local as the best current fit for this deployment.
The pinned TP2/DCP2, FP8-KV, 327,680-token/C4/MTP3 configuration is promoted;
EXL3 r11 is retained as rollback. The matched two-recipe comparison measured
faster C1 prefill and decode, with mixed concurrent latency and narrower GPU
headroom. The promotion record
explains the selection, client acceptance, KV accounting, and remaining fleet
workflow blocker. The full comparison
retains every measured cell and failure; this preference does not establish
an exhaustive quality ranking.
2026-10-15 replacement evaluation: the completed campaign finding records no eligible challenger. DeepSeek V4.1 Flash remained bounded to diagnostic gates, and exact GLM was restored and retained. The finding is the detailed gate record; this does not add a new GLM qualification, route change, or performance ranking.
Historical bounded recommendation, 2026-10-13: The retained recommendation was the non-abliterated official-base GLM TR3 EXL3 4-bpw R11 baseline: TP2/EP2/DCP1, FP8 DS-MLA KV, 327680 configured tokens, C4, APC, no speculation, and vision 8. R19 accepted exact restoration with direct 13/13, routed tools/SSE/JSON_OBJECT and vision/OCR, Pi settled read, Hermes, OpenClaw high 9/9, and Hindsight recall/retain. Thinking-off remains failed and unified evaluation is not blanket green. The fresh fixed-five baseline is 2/5; retention does not claim a universal quality winner or a new full-context C4 measurement. Exact configuration and evidence.
2026-10-13 full qualification: campaign closure retains separate GLM, Qwen, DeepSeek, MiMo, and Ignis evidence. The GLM baseline completed 2/5; terminal Ignis was incomplete at 2/5 resolved with three empty cases. No challenger cleared the complete 4/5 plus Django/Pylint requirement, so no challenger was promoted.
Eight-image follow-up, 2026-09-18: GLM EXL3 r9 passed four eight-image comparison requests, one eight-image high-resolution request. Limit now eight/request; no concurrent eight-image soak.
Historical 2026-09-23 publication-verified decision: GLM DCP1 r11 is selected and deployed after retained direct/routed and native-client acceptance. Independent reviews, dashboard receipt, successful Pages publication, and HTTP/browser readback are retained. Exact r10 is the rollback.
2026-10-05 qualification decision: Qwen replacement and GLM CSF/QAD evidence rejects Qwen at 3/5 and GLM after fresh-reload A1 resolved 2/5 attempts (2/4 graded), below the 4/5 floor. GLM's earlier 4/5 scout, functional 25/25, Oracle 18/18, vision, and bounded context acceptance remain retained but do not establish a reliable quality floor, full-window capacity, or speed. B1/B2/A2 did not run. Exact r11 DCP1 restoration completed with direct 25/25, routed 7/7, and a fresh Pi read-tool turn; independent review follows closure.
Compare local models by useful context, quality, and serving performance. Start with the latest decision, choose your hardware, then follow the exact configuration and retained evidence. Evidence reviewed: 2026-11-01 Swift TP1 qualification and separate private-target runtime scouts; publication readback remains partial.
Engine testing¶
The October 25 sampling-fix trial stopped at a synthetic ordinary-mask parity failure before model weights or inference. It supplies partial functional kernel evidence, no candidate promotion or proven model-boundary fix.
The engine-testing reports collect the closed Flare diagnostics and the 0.7.2 Ignis qualification. They are bounded local evidence: the incumbent remains selected, no Flare result is promoted, and Ignis coding qualification is incomplete. Use the reports for their exact evidence and failure boundaries, not as live deployment status.
Historical bounded APC reference — September 19¶
Human-approved GLM-5.3-Flash r10 APC · Dual RTX PRO 6000
The bounded r10 APC configuration retains the 4-bpw GLM checkpoint, v84 runtime, FP8 MLA KV, no speculation, 327,680 configured tokens, C4, and the eight-image contract. On frozen 32-request finalists with about 25K actual prompt tokens, it improved the shared-prefix path while staying within the unique-prefix gate.
| Matched outcome | r9 | r10 APC |
|---|---|---|
| Shared-prefix visible TTFT | 23.70 s | 7.62 s (−67.84%) |
| Shared-prefix requests/s | 0.146 | 0.462 (3.17×) |
| Unique-prefix requests/s | 0.144 | 0.141 (−2.39%) |
The 75.58% isolated cache-reuse fraction supports the intended repeated-prefix mechanism. This does not prove four simultaneous maximum-context requests, video readiness, a soak result, or broad intelligence/SWE parity. Roll back to the exact r9 recipe if correctness, containment, routed readiness, or the frozen unique-prefix gate fails.
Read the promotion follow-up · Inspect the historical campaign · Open the matched chart
September 14 quality and performance baseline¶
Historical r7 text and tool qualification · Dual RTX PRO 6000
GLM-5.3-Flash · EXL3 4 bpw · speculation off¶
The September 13–14 qualification selected this configuration after bounded quality, context, and client tests.
| What was tested | Result |
|---|---|
| Context / output capacity | 327,680 / 65,536 tokens |
| Long-context retrieval | 9/9 twice, including after cold reload; 255,647–255,672 input tokens plus output reserve |
| Quality / agentic / repository tasks | 90/100 fixed MMLU-Pro sample · 30/30 agentic · 4/5 SWE tasks |
| Controlled requests / decode | 120/120 · 38.91 tok/s median per request at C4 |
Limits of this baseline: tested text only; no exhaustive intelligence ranking, prolonged soak, reboot, or simultaneous full-window proof. These are dated local results, not a live status display or an automatic model-selection policy.
Read the qualification · Open the model dossier · Inspect the evidence
2026-09-24 secondary promotion: ThinkingCap Qwen3.8 27B AWQ is current after the human-authorized promotion of its qualified RTX 5090 128K/C1 profile, with vision and complete local weights. The direct qualification retains 27/27 functional, 9/9 thinking, 18/18 small-image, 30/30 limited quality, 100/100 strict short-output capacity, 60/60 text-context and 2/2 synthetic agentic passes. The selected recipe measured 3,003.045 ms median TTFT and 51.885 tok/s median decode on the controlled short-output workload; the faster qualified 32K alternative measured 455.442 ms and 158.355 tok/s. These compare whole recipes. The final routed contract is 131,072 total / 10,240 output / C1 / two images. Pi tool/result acceptance and managed catalog convergence passed; Open WebUI browser chat, detailed remote throughput monitoring and the intermittent image relay cause remain open. Mixed near-limit vision, C2, exact-limit/one-over, broad SWE and soak are unqualified.
Choose by hardware and workload¶
1. Pick your hardware
Start with the measured RTX PRO 6000, RTX 5090, or Apple M4 Max view. GPU count, interconnect, operating system, and runtime are part of the result.
2. Name the workload
For interactive use, prioritize TTFT and E2E. For sustained serving, use aggregate output, TPOT/ITL, and tail latency. For long context, require actual prompt depth rather than configured context.
3. Check the tradeoff
Use measured recipe results for explicit strengths, limitations, and rejection reasons. Use the model and quantization selection guide for a bounded workload choice, then the complete model inventory when quality, tools, media, or client acceptance matters more than one speed number.
4. Reproduce and verify
Open the exact recipe, then follow its dated finding and native JSON artifact. A copied container configuration is not a reproduced result until the same workload and independent gates pass.
Start with comparable numbers¶
For speech recognition, speech generation and spoken answers, start with voice model results. The latest Apple comparison retains all failed gates and no-promotion decisions alongside the measured results.
The model comparison table puts the wider measured configuration — TTFT, throughput, context, reasoning mode, and recipe link — in one place, grouped by card and workload. The measured recipe finder is the shorter path when you want a current configuration, its tradeoffs, and raw evidence.
For the newest single-card Qwen measurements, the dedicated Qwen3.8 27B RTX 5090 quant comparison shows every fresh no-speculation/speculation pair, full-context proof, and failure without compressing the operator's TTFT-first ranking into one score. The newer dual-PRO Qwen3.8 27B comprehensive campaign compares SGLang target-only/DFlash2 tuning, TP1/TP2/two-replica DP2, Inferact/RadixArk targets, and kelnei/vLLM MTP2/no-spec. It retains mean, p50, p95, and p99 TTFT, effective prefill, decode, TPOT/ITL, and E2E plus aggregate throughput and correctness failures. Open the campaign graph (SVG) or its plotted data and source hashes.
For the compact publication contract, use the finding format. It defines screenshot-ready result cards, the common campaign artifact-set template, X and Reddit variants, accessible alt text, and a claim ledger back to the full finding and raw artifacts.
Browse by hardware¶
| Hardware | Published evidence scope | Start here |
|---|---|---|
| 2× NVIDIA RTX PRO 6000 Blackwell Max-Q, 192 GB aggregate, sm_120 | Measured dual-card reference configuration; split workloads or exclusive TP=2 | RTX PRO 6000 |
| NVIDIA GeForce RTX 5090, 32 GB, sm_120 | Measured single-card qualification configuration, plus retained historical results | RTX 5090 |
| Apple M4 Max laptop, 48 GB unified memory | Historical same-host voice candidate evidence; no LLM promotion or reference-topology change | Apple M4 Max |
The published dual-PRO reference configuration uses two equal PRO 6000 cards. Aggregate VRAM is not unified memory, and the cards communicate over PCIe without NVLink. Exclusive TP=2 runs must prove that both devices were selected and all other inference was offline. The published reference topology keeps Companion Node model-free. Historical mixed-card tests still say which card was measured and which was merely protected or co-resident.
Browse by model¶
The complete hardware-first inventory lists every dossier and explicit grouped variant, including the September 12 efficient Qwen models and the Apple MLX Qwen lanes. Each dossier is a stable, model-centered summary with its dated findings and raw evidence.
Recent evidence¶
-
2026-10-06 — Apple English voice: quality-first component qualification retains all STT/TTS/LLM failures and no promotion. Human-only STT chart, complete and resource-aborted TTS arms, separate off/low / medium LLM profiles and bounded restoration are auditable; failed strict 8K cells are not performance ranked. Hardware · all runs.
-
2026-09-15 — media: The media finding records bounded workflow evidence; use its dossier and finding rather than treating it as a text-model selection result.
- 2026-09-12 — efficient Qwen variants: Signal, Swift, Qwopus, and Minitron all passed functional checks on the RTX 5090, but none cleared the exact incumbent's output and deployment gates. See the grouped dossier.
Run context, agentic, and repository evaluations¶
The context, agentic, and SWE job guide describes the registered-worker workflow, bounded profiles through 640K, independent scoring, pinned mini-SWE-agent and official SWE-bench grading, cancellation, and the measured-versus-prior evidence boundary. The guide defines methodology and commands; it does not claim a live model result by itself.
Published decisions and recent controls¶
The collapsed history below preserves earlier decisions through September 11.
Labels such as current and rollback describe their status at that date;
the September 13–14 selection above supersedes the primary-model guidance.
Use the run catalog to compare dated configurations.
Earlier decisions and controls — historical configurations
How to read the evidence¶
Evidence labels describe what was observed:
| Label | Meaning |
|---|---|
external-prior |
Official or community research used to choose a recipe; not local proof. |
compatibility-only |
Loaded or answered bounded probes; no qualification claim. |
functional |
Independent behavior gates passed for the stated contract. |
capacity |
Context, concurrency, throughput, latency, or residency was measured. |
quality |
A declared quality workload was measured with retained results. |
historical-invalid |
Retained run is incomplete, incomparable, or missing identity needed for reuse. |
Decision labels are separate: current, rollback, challenger,
no-promotion, and rejected. In this public documentation, they describe the
latest published evidence decision for a reference configuration, not live
deployment state. A quality result can still be no-promotion; a failed load
can be rejected while remaining useful compatibility evidence.
Full comparison rules, instrument definitions, and artifact requirements: Methodology and evidence rules.
Complete history¶
- Run catalog — every retained, decision-relevant run, indexed by hardware and date.
- Findings index — the dated reports with full commands, raw artifacts, and failure cases.
- Finding publication format — compact result and copy formats without weakening evidence boundaries.
- External benchmarks — advisory priors imported from official and community sources, kept separate from local results.
- Chronological campaign archive — the stable-URL summary of historical rounds, including Fast-tier and voice campaigns.