GLM-5.3-Flash v0.9.1 trial¶
Date: 2026-10-16
Status: Closed evidence record. Fresh matched incumbent Oracle controls completed 54/54, matching vision completed 18/18, matching long-context probes passed at 216307/110760 actual prompt tokens, and the matched SWE control completed 3/5. Closing telemetry confirms exact original restoration, zero OOM, and removal of both candidate containers.
Decision: R2 is rejected and no-promotion. Its 3/5 SWE result missed
the declared four-case replacement gate. Exact incumbent restoration is
verified; no route, client, or provider setting changed.
Result card¶
On two RTX PRO 6000 Blackwell Max-Q GPUs, the exact v0.9.1 R2 profile passed functional, Oracle, vision, context, and sampled reserve gates but failed the bounded coding replacement gate at 3/5.
| Setup | Recorded configuration |
|---|---|
| Model | brandonmusic/GLM-5.3-Flash-tr3-4bpw@a5fee929cf4888b1824323e33e8a19b60129e025 |
| Runtime | v0.9.1 image sha256:a9e985c09885710601b32ac86e81256aa1279b542caa345233ce331a816d7e07 |
| Contract | EXL3 4 bpw, FP8 DS-MLA KV, TP2, 327680 configured tokens, C4, vision eight, no speculation |
| Evidence | functional, quality; fresh incumbent control completed 3/5 |
| Decision | rejected, no-promotion; exact incumbent restored |
| Headline measurement | Local result | Conditions |
|---|---|---|
| Native preflight | 6/6 | R2, default thinking/max effort |
| Oracle | 54/54 | three strict 18-case R2 rounds |
| Vision | 18/18 | 18 checks; up to eight images per request |
| Coding | 3/5 | pinned SWE suite; four required |
| Sampled reserve | 4789 / 5583 MiB | 200 ms samples; 4096 MiB floor |
Important caveat: the matched control and R2 each resolved 3/5, below the four-case gate. Their matching resolved counts do not establish equivalent grader failure mechanisms, causal behavior, or performance. Capacity, stability, DFlash, and performance cells remain ineligible.
Evidence index: sanitized bundle.
Stable outcome¶
R1 passed native preflight 6/6. Its temperature-one Oracle first round failed strictly at 17/18; the frozen temperature-zero sampling remedy then passed 54/54, vision 18/18, and retrieval/tool-context at 216307/110760 actual prompt tokens. Its sampled reserve then fell to 3879 MiB, below the 4096 MiB floor, so the SWE run was cancelled with partial evidence retained. R2 lowered GPU memory utilization from 0.95 to 0.94 and repeated the prerequisite tests. It passed native preflight 6/6, Oracle 54/54, vision 18/18, the same context probes, and the sampled reserve gate with zero OOM events. Its completed official SWE grade resolved 3/5; Django Unicode JSON display and Pylint regex handling failed. The candidate did not proceed to capacity, stability, DFlash, or a performance comparison.
This rejects only the evaluated R2 profile under this bounded workload. It does not establish a runtime or model-family defect.
Restoration¶
The original GLM recipe was restored with the same captured container image,
model, and revision. Direct and routed native preflight both passed 6/6, and an
actual Pi read turn reached agent_settled without persistent provider changes.
Container identifiers, endpoints, host names, and private paths are retained
only in protected source evidence.
Disabled-thinking strict format checks failed for both the baseline and candidate because reasoning appeared in visible output. The retained default-thinking/max-effort mode is the matched operating mode; this failure is not silently rescored as a pass.
The 4096 MiB reserve is an R2 challenger gate. The exact restored original retains its historical recipe waiver; its control telemetry must not be labeled a challenger-gate pass or used to retrofit a new restoration policy.
Matched incumbent control¶
The exact restored original passed fresh direct/routed preflight 6/6, Oracle 54/54 at temperature zero, vision 18/18, and the same retrieval/tool-context probes at 216307/110760 actual prompt tokens. Its matched five-case SWE control completed and resolved 3/5: Django and Pylint remained unresolved, while Xarray, Sphinx, and Sympy resolved. R2 resolved the same cases, also 3/5 and below the four-case gate. Both failed Django Unicode JSON display. Pylint had two failing tests and 18 passing tests for R2, versus one failing test and 19 passing tests for the original. The original failed invalid-regex error handling; R2 additionally failed regex-quantifier comma handling. These detailed observations do not establish a general quality difference or its cause.
Controls ran after the candidate was quiesced and router admission was restored. The only comparison recorded here is the bounded per-instance and resolved-count result; it supports no causal, throughput, latency, or model-family conclusion. Historical 4/5 SWE evidence is not used as this campaign's baseline.
Evidence boundary¶
The bundle preserves fixture prompts, bounded model observations, validators, sanitized native stage output, source hashes, and redaction provenance. Full trajectories, patch text, grader command files, unsanitized operational logs, operator paths, and machine identity remain protected. The provenance record identifies retained source objects and protected grader reports. No compact public post was requested.