2026-10-16 GLM natural-output baseline evidence¶
This is a closed public-safe baseline bundle for the current GLM-5.3-Flash r11 route. The timing matrix is retained and coding repeatability was not established. It does not support a qualification, replacement ranking, or promotion decision.
Closed evidence index¶
The common manifest binds every retained file to its size, digest and role. Its source inventory contains all ten roles and no pending artifacts.
- Plan and identity: run plan, workload manifest, configuration pins, source registry, and coding amendment.
- Timing: summary, summary provenance, source hashes, graph manifest, SVG chart, and graph data.
- Coding R1: native SWE run, official grader report, and aggregate campaign extraction. R1 resolved 4/5 with complete independent grading.
- Coding R2: native SWE run, official grader report, and failed campaign extraction. R2 resolved 0/2 graded patches; three empty step-limit attempts were ungraded. The failed aggregate extraction has no score.
- Interpretation: final coding summary, independent R2 review, and decision summary.
- Failures, continuity and limits: friction log, errata, and serving continuity with open storage limitation.
- Provenance: redaction ledger and SWE source/public bindings.
The twelve native timing artifacts retain their original metrics and request
fields. Only the routed endpoint is replaced with the non-routable
redacted://routed-capability value.
| nominal context | concurrency | repeat 1 | repeat 2 | repeat 3 |
|---|---|---|---|---|
| 8,192 | 1 | r1 | r2, retained failure | r3 |
| 8,192 | 4 | r1 | r2 | r3 |
| 32,768 | 1 | r1 | r2 | r3 |
| 32,768 | 4 | r1 | r2 | r3 |
Native SWE stages and official grader reports are authoritative per task. The aggregate extractions preserve the existing wrapper evidence and do not add per-task comparison semantics. Private wrappers, trajectories and raw process output remain private. Task-image tags and immutable digests are retained at each image observation point.
Boundaries¶
Natural output has variable length. Completion-token rates can include reasoning tokens and are not controlled-decode or visible-only speed. The router was shared and the workload snapshot was unavailable, so these are not traffic-isolated or engine-only measurements. Each timing cell has 20 requests; three repeats may describe repeat-median ranges but cannot establish stable p99 latency or service stability. The retained matrix has one no-visible-output failure and two long C1/32K reasoning outliers; neither is filtered.
The original plan stated that finish reasons would be retained. Successful
native timing rows retain visible-output and terminal-observation timing, but
not finish reasons, terminal form, raw usage objects, or visible character
counts. errata.md records this correction.
The coding baseline now uses the existing pinned five-task set twice because
the fresh ten-task images are uncached and storage reserve is constrained. It
is a repeatability observation, not a fresh held-out SWE score; see
coding-amendment.md.
All planned attempts are exhausted; grading is incomplete for R2 and coding repeatability was not established. Full SWE-interval server logs and per-retry transport telemetry are unavailable. Serving continuity was verified, but the post-run storage check was below the 100 GiB admission reserve. The cause is unproven, no pruning was performed, and further trials require that separate storage gate to be resolved.