Skip to content

Qwen3.8 efficient fine-tunes and Minitron

Current status and review date

November 1 single-GPU TP1 Swift qualification

The BF16/FP8-KV Swift 1.5 Qwen3.8 27B recipe passed bounded functional, context, image, agentic, SWE, stability, resource and capacity gates on one RTX PRO 6000 Blackwell Max-Q. The fresh ten-cell population is separate from both the historical 60-cell no-spec/MTP comparison and the RTX 5090 result. C2 trades per-request speed for aggregate throughput; its 32K median decode missed the soft objective (13.43 versus 15 tokens/s). C1 remains latency-first. Strict formatting failed 4/4 and the SWE pilot resolved 4/5. Current deployment, routing and client assignments are private. Finding.

October 5 host-cache promotion

The retained Swift-1.5/NInfer NVFP4 derivative is a separate configuration from September's Q6_K challenger. On one RTX 5090 with nominal 96 GB host RAM it passed three preflight repeats, agentic 18/18, context 36/36, images 12/12 and five-case official SWE 4/5. Strict repeated-word capacity failed 0/12; a separate variable-output service workload passed 48/48 across four 8K/24K C1/C4 cells. A separate immutable-image SWE replay also resolved 4/5. The initial Flash Next 12-request cell failed the practical latency gate against this baseline; instrumented diagnostics do not replace it. The same-image 8 GiB/eight-slot host-cache pilot passed: 12 revisits per arm retained about three times more tokens, with median visible TTFT 9.100→5.353 s and E2E 9.227→5.498 s (average-middle-pair convention). Fresh service, agentic, long-context, image and explicitly cache-aware continuation gates passed; final configuration and client acceptance passed; the 8 GiB/eight-slot profile is promoted. Evidence cutoff/review: 2026-10-05. Finding and evidence. The snapshot below retains September's decision.

Decision snapshot

  • Product role: direct RTX 5090 replacement candidates, not promoted services.
  • Selected or best-qualified configuration: no qualified replacement; tested Signal and Swift 27B Q6_K/MTP3, Qwopus Flash 27B Q6_K/no-spec, and Minitron 20B Q6_K/no-spec; all 64K/C1.
  • Measured hardware: one NVIDIA RTX 5090, 32,607 MiB, Windows/Docker/WSL2.
  • Evidence: functional passes for all four; bounded quality and failed controlled-output scouts. These are not broad benchmark accuracy scores.
  • Decision: retain the exact incumbent; tested challengers no-promotion.
  • Important limitation: no challenger has qualified the incumbent's 262K deployment contract or the declared complete-output performance gate.
  • Review dates: evidence cutoff and review 2026-09-12.

Managed recipes, no-spec controls, decision, and complete evidence index.

Review narrative

The four candidates fit and served the functional workload. Their efficiency claims did not remove strict output or bounded quality failures. The exact incumbent has been restored and passed its post-run checks. It scored 8/10 thinking-on and 21/30 thinking-off, versus Signal's 9/10 and 24/30. Signal is the best new efficiency research lead, but not a qualified full-contract replacement. The incumbent was faster on the revised cold short-output scout.

Immutable identity

October 5 Swift15 NVFP4 baseline

kaushikvira/Qwen3.8-27B-swift15-nvfp4full-dflash2-NInfer-v3 at ff891a1130adfcf46bf745e06f0fa3dcde2e8c82; NInfer bace20dc70249eed6402b66d4852c6c3f9612905, runtime image sha256:59df48dd99f0d177b73de57ba88eb6457ed7e4b0c5771c45e8976adcd9f5ef67. Exact reconstruction parameters retain model-file/recipe hashes, K8V4, DFlash2 draft7, 262K/C4, 2 GiB host KV and four host-state slots. Context/quality cells are serial; short service latency separately exercised C4. This does not inherit September's evidence.

September candidates

Recipe family Weight repository Exact revision
Signal 27B agentionai/Signal-3.8-27B-GGUF 9848da210febc2141edb7e3f89cfc307a0d15163
Swift 27B ukisai/Swift-Qwen3.8-27B-GGUF dfc5e7382fa86bd3b971108e23423be5ef18941e
Qwopus Flash 27B mradermacher/Qwopus3.8-27B-Flash-i1-GGUF a2f33e11aa22e467206633d7af9da1990f7111b9
Minitron 20B mradermacher/Qwen3.8-20B-Minitron-i1-GGUF ef2d8cbbb83bbe27142546665163d2ab9d31727c

The source registry separates upstream fine-tunes from quantizers and records model-card priors. Swift's pinned source license has a commercial revenue threshold; eligibility was not assumed. Signal and Minitron cards identify Apache 2.0.

Tested hardware and topology

One RTX 5090 (sm120), driver 616.64; direct loopback endpoint under Docker/WSL2. The incumbent was retained stopped while one candidate ran at a time. The pre-existing Windows camera utility was protected. No other GPU was measured. The router was absent at baseline; no client catalog or route was changed.

Engine, quantization, KV, context, and concurrency recipe

All four use llama.cpp source a298422da78eb75e440a7de0ca408af64d323d93, CUDA server image digest sha256:cf2e30bc855cf58cdbdc65d05b5b5e02afa95fb788343a5334d704367ac5c9ac, Q6_K weights, Q4_0 K/V, full GPU layer placement, Flash Attention, 64K context, C1, 512/256 batch/microbatch, eight threads, Jinja, and per-request thinking control. Signal/Swift have matched no-spec variants. Exact filenames, sampling, graph settings, and flags are in the linked recipes.

Evidence by measurement class

October 5: recovered baseline on larger system RAM

  • Status: functional and bounded quality/context; failed strict stress and a separately versioned variable-output service-latency population.
  • Measured: three preflight repetitions passed, agentic 18/18, context 27/27 plus 9/9 at 253,838–253,868 actual inputs with 8,192 reserve, images 12/12, official SWE 4/5 of five fixed tasks. Service-v2 passed 12/12 per cell.
  • Limits: no candidate speed comparison, fixed-output decode ranking, broad coding score or full C4 context envelope. Historical SWE image identities are boundary-only observations. Strict repeated-word failure does not imply ordinary prose streaming failure.
  • Evidence: baseline checkpoint.

Four-model functional and quality scout

  • Status: functional, diagnostic quality, failed strict capacity.
  • Measured: every original candidate passed all six preflight checks, including tools 20/20 and exact retrieval at 47,349 actual prompt tokens. Thinking-enabled ten-question strict scores: Signal 9/10, Swift 9/10, Qwopus 1/10, Minitron 3/10.
  • Limits: scores count answer-format and budget failures; one repetition cannot support general knowledge accuracy or a 2% quality-loss bound.
  • Evidence: raw runs and interpretation.

Signal and Swift both exhausted the same computer-science question at 5120 total completion tokens. Qwopus had eight format failures and one exhaustion; disabling thinking improved strict repeated results to 24/30 but did not remove budget failures. Minitron passed 15/30 with thinking disabled and used 24,580 completion tokens in the enabled scout, versus 10,966/11,020 for Signal/Swift. Smaller did not mean less reasoning in this bounded sample.

Controlled-output failures and diagnostic repair

  • Status: negative capacity, not a speed ranking.
  • Measured: Signal and Swift completed 0/5 exact 128-word requests with MTP and again without it. Swift no-spec and Qwopus each passed 5/5 revised 32-word requests; Minitron completed 0/5.
  • Limits: original failures remain. Swift's revised population was warm while Qwopus was cold; their timings are not a matched comparison. No speculation speedup is established. N5 p99 is descriptive only.
  • Evidence: failure ledger.

Decision and promotion state

Retained or selected

The operator authorized unattended best-supported selection and promotion. That authority does not turn a failed gate into qualification. Keep the existing 262K Unsloth GGUF/MTP3 deployment: exact restoration and all six post-run functional gates passed. No route or client catalog changed.

Rejected or incomplete

The four tested 64K profiles remain no-promotion; none is a qualified 262K replacement. Qwopus and Minitron have additional quality failures. Signal is an efficiency research lead, not a proven full-contract upgrade.

Failures and gotchas

  • Capacity calibration:57344 nominal filler produced 47,349 actual tokens; do not call that a57K-plus8K qualification.
  • Invalid speed: repetitive cap-hitting responses are failures, not useful decode throughput. Removing speculation did not repair 128-word adherence.
  • Quality interpretation: strict syntax, visible budget, hidden reasoning, and factual correctness are separate failure causes. The ten-question fixture is diagnostic and not a representative MMLU-Pro evaluation.
  • Timing contamination: repository tests overlapped some diagnostic quality collection; those wall times are not clean finalist latency.
  • Lifecycle tooling: retained-container restoration required guarded product fixes. Independent review accepts serialized campaign use but holds general release pending shared lifecycle locking and immutable process attestation. The separate Windows Pi transport fixes passed a real smoke.

2026-09-26 Swift 1.5 Flash Next TP2

  • Status: bounded observations; SWE fixed denominator 2/5 failed the 4/5 floor. No qualification or promotion; exact GLM incumbent restored on the current host.
  • Measured: exact NVFP4 TP2 r3 passed protocol/tools, agentic 18/18, context 12/12 at ~32.7K and ~256.9K prompts, images 12/12, and eight images 4/4.
  • Limits: C1 only; configured 327,680 context plus a requested 65,536 output allowance does not prove an actual full-window prompt or full output. No matched baseline, throughput, or cost result exists.
  • Decision: retain GLM; no route or promotion change. See the dated finding.

Dated run history

  • 2026-10-05 — Swift15 upgraded-host baseline: bounded correctness and quality; strict stress ineligible; separate service-latency cells retained; 8 GiB/eight-slot host-cache pilot passed with more retention and lower revisit latency; fresh regression and explicit-history gate passed; final client acceptance passed and the expanded cache profile was promoted.

  • 2026-09-26 — single-GPU Swift qualification: Flash IQ3 r3 passed deterministic gates but SWE 3/5; Swift27 NVFP4 r2 resolved 2/5 with four graded. Neither is qualified or promoted.

  • 2026-09-19 — Apple artifact feasibility stop: the separately pinned Swift/stock GGUF families were not downloaded or run. All selected uncached Q6_K/Q4_K_M plus F16-projector pairs failed the retained disk-policy arithmetic; memory containment was unresolved. This is not a quality or performance result and does not change the RTX 5090 candidate decision.

  • 2026-09-12 — Efficient-variant finding and complete evidence.