Measured recipe library

These are the exact container contracts behind the most useful retained benchmark results: pinned model revisions, image identities, cache mounts, environment controls, engine flags, context, concurrency, and speculative decoding controls. The full TOML below is included directly from the tracked recipe files, so this page changes when the source recipe changes.

Managed operation and independent reconstruction

Anvil Serving operators should use the managed models recipes and serves lifecycle instead of raw Docker. The field mapping on this page is also provided for readers using another container runner. In that case, you own device selection, cache preparation, health checks, teardown, authentication, and resource isolation; reproducing a container does not reproduce a benchmark result or authorize a route change.

For a decision-first view, open Find a measured recipe. It reads metrics from the native artifact, keeps strengths beside limitations, and links back to the exact TOML, dated finding, dossier, and raw evidence.

Configuration index

Configuration family	Recorded hardware and runtime	Served envelope	Exact source
Qwen3.8 27B September 2026 PRO 6000 optimization family: Inferact finalist/TP2 control, RadixArk TTFT tradeoff, and kelnei vLLM MTP2/control	1–2× RTX PRO 6000 Blackwell Max-Q, WSL2; digest-pinned SGLang and vLLM 0.27.1	262,144 tokens; C8 per service; matched 4K/C8 sustained-output evidence	Open family
GLM-5.3-Flash EXL3 K3 + DFlash2 K5 and matched no-spec control	2× RTX PRO 6000 Blackwell Max-Q, WSL2, vLLM-derived local image, TP=2/DCP=2	524,288 tokens, max 16 sequences, up to 16 images, no video	Open pair
GLM-5.3-Flash ormandj W4A16/NVFP4 SGLang adaptive MTP and control	2× RTX PRO 6000 Blackwell Max-Q, WSL2, digest-pinned SGLang rc14, TP=2	current 393,216-token C1 lane; 245,760-token fallback, 131K A/B, and 499K negative controls retained	Open family
Qwen3.8 Flash Next RadixArk NVFP4 MTP3 and matched no-spec control	2× RTX PRO 6000 Blackwell Max-Q, WSL2, digest-pinned SGLang with hash-gated SM120 patching, TP=2	262,144 tokens, one running request	Open pair
Qwen3.8 27B official FP8 SGLang MTP3 multimodal and no-spec campaign control	1× RTX PRO 6000 Blackwell Max-Q, WSL2, digest-pinned SGLang, TP=1	393,216 tokens, one running request, CPU media transport	Open pair
Qwen3.8 27B RadixArk NVFP4 multimodal	1× RTX 5090, digest-pinned SGLang, TP=1	131,072 tokens, one request, up to eight images or two videos	Open configuration
GLM-5.3-Flash EXL3 K3 + DFlash2 K5, same-model rollback	2× RTX PRO 6000 Blackwell Max-Q, WSL2, Purtell-derived vLLM image, TP=2/DCP=2	1,048,576 tokens, max 16 sequences, image/OCR	Open inventory
DeepSeek V4 Flash 0731 Infernal Invocation r18 DSpark K5 and no-spec control	2× RTX PRO 6000 Blackwell Max-Q, WSL2, custom r18 vLLM image, TP=2/DCP=1	1,048,576 tokens, max 8 sequences, text	Open inventory

Decision terms such as current, rollback, and no-promotion describe the latest published evidence decision, not an operator's live deployment.

Translate the recipe to a container runtime

Anvil Serving's recipe loader turns these fields into a container invocation. An independent runner can preserve the same contract with this mapping:

Recipe field	Container-runtime meaning
recipe.download.repo + revision	Fetch this exact model snapshot. A branch name or latest revision is not equivalent.
recipe.download.volume	Named model-cache volume, mounted at /root/.cache/huggingface by Anvil Serving. Populate the exact snapshot before an offline load when require_complete_cache = true.
recipe.download.require_complete_cache	When true, Anvil Serving also injects HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1. An independent invocation must add both values even when they are absent from recipe.serve.env.
recipe.hardware	Hardware prerequisite and topology assumption, not a portable device selector. Select local devices that satisfy the GPU count, architecture, VRAM, and TP/DCP shape; never copy a host UUID from someone else's machine.
recipe.serve.image	Exact container image. Preserve @sha256: when present. A local image name requires the recorded build assets and a digest check before it is equivalent.
entrypoint	Replace the image entrypoint with the listed executable and arguments. Some recipes apply guarded source changes before launching the server.
model_flag, model_path, model_env	Supply the immutable in-container snapshot either as a command flag, positional argument, or environment value, exactly as declared.
named_volumes	Additional type=volume mounts for generated kernels, runtime caches, or temporary state. These do not replace the separate Hugging Face model-cache mount.
env	Container environment variables. Preserve allocator, NCCL, WSL2, feature-transport, parser, and cache controls; they are part of the measured configuration.
flags	Server arguments after the image entrypoint. Preserve quoting for JSON-valued flags and keep context, KV dtype, concurrency, batching, parsers, and speculation together.
port	Container and loopback host port. Anvil Serving publishes it on 127.0.0.1; do not turn an internal --host 0.0.0.0 into an unauthenticated LAN or internet bind.
ipc, shm_size, ulimits	Container IPC, shared-memory, and process-limit settings. Omitting them can change startup or long-context behavior.
status, note, fit, sources	Evidence and provenance metadata. They do not become container arguments, but they define the known limits of reuse.

Assemble the equivalent container without Anvil Serving

You need only Docker, a TOML reader, and access to the declared model and image artifacts. Work from a local copy of one full snippet below:

1. Pull or build recipe.serve.image and verify its digest. Stop if a local or custom image cannot be recreated from the linked assets.
2. Create recipe.download.volume, populate the exact recipe.download.repo@revision snapshot, and mount it at /root/.cache/huggingface. When require_complete_cache = true, also add HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1. Create every named_volumes source separately.
3. Select local GPUs matching recipe.hardware and the declared TP/DCP shape. Substitute your own <GPU_UUID_A> and optional <GPU_UUID_B>; no public recipe supplies them for you.
4. Build the container argument vector in this order: runtime controls, environment, cache mounts, named mounts, loopback port, IPC/shared-memory/ ulimits, entrypoint, image, model selector, then engine flags.
5. Inspect that argument vector before launch. Confirm the model revision, image digest, mounts, GPU count, loopback publication, and served name, then start it with no unrelated GPU workload sharing the declared devices.

The shell-neutral layout is:

docker run --name <LOCAL_CONTAINER_NAME>
  --gpus <GPU_REQUEST>
  -e CUDA_VISIBLE_DEVICES=<GPU_UUID_A>[,<GPU_UUID_B>]
  -e <EACH recipe.serve.env ENTRY>
  [-e <recipe.serve.model_env>=<recipe.model> WHEN model_env is present]
  [-e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 WHEN require_complete_cache=true]
  -v <recipe.download.volume>:/root/.cache/huggingface
  --mount type=volume,source=<NAMED_VOLUME>,target=<ABSOLUTE_TARGET>[,readonly]
  -p 127.0.0.1:<recipe.serve.port>:<recipe.serve.port>
  --ipc <recipe.serve.ipc>
  --shm-size <recipe.serve.shm_size>
  --ulimit <EACH recipe.serve.ulimits ENTRY>
  --entrypoint <recipe.serve.entrypoint[0]>
  <recipe.serve.image>
  <REMAINING ENTRYPOINT ITEMS>
  <recipe.serve.model_flag> <recipe.serve.model_path>
  <TOKENIZED recipe.serve.flags ENTRIES>

For one device, <GPU_REQUEST> is the argument device=<GPU_UUID_A>. For two devices, Docker must receive the literal argument "device=<GPU_UUID_A>,<GPU_UUID_B>", including the inner double quotes, because its --gpus parser treats commas as capability-request separators. Escape or quote that literal for the shell or container API you use; shell syntax is not portable across PowerShell, Bash, and structured argv calls.

Omit only fields absent from that recipe. The explicit device selection also becomes CUDA_VISIBLE_DEVICES, unless the recipe opts into a validated container-relative numeric index pin. When model_env is present, pass the model through that environment variable and omit the model argument. Without model_flag or model_env, append model_path (or recipe.model) as the positional model argument. Preserve each JSON-valued flag as one argument. The recipes with embedded multi-line entrypoints are intentionally not emitted as copy-paste commands here: PowerShell, Bash, Compose, and container APIs quote those values differently. The checked-in TOML plus this ordered mapping is the complete, shell-independent reconstruction contract.

The model cache and each named volume must exist independently. Volume names are convenient labels, not downloadable artifacts. Generated kernels, JIT caches, or temporary volumes may start empty only when the image can recreate their contents; the pinned model snapshot may not.

Reuse boundaries

Hardware, images, and licenses are part of the result

These configurations were measured on sm_120 Blackwell GPUs. The dual-card recipes assume separate PCIe devices without NVLink and the recorded WSL2/NCCL controls; TP=2 is sharded capacity, not unified memory. Another GPU, native Linux, another CUDA/driver build, or a different GPU count is a new configuration requiring fresh evidence.

The rollback GLM 524K image is a local Anvil-derived build. Recreate it from the tracked Dockerfile and xgrammar patch, then verify the recipe's image digest. The Qwen3.8 Flash Next pair uses an upstream image but performs exact, hash-gated source edits in its recorded entrypoint. The DeepSeek r18 image is a community-derived runtime whose launcher scripts are embedded in that image. Those recipes are not standalone when the exact image or required build inputs are unavailable.

DFlash2 is licensed CC BY-NC-ND 4.0 and remains limited to evaluation and noncommercial use without separate permission. Model weights, images, and other dependencies retain their own upstream licenses; this recipe index grants no additional rights.

The public recipes intentionally omit physical GPU UUIDs, private host addresses, credentials, and active route assignments. Supply those locally; do not add them to a public reproduction report.

Qwen3.8 27B September 2026 RTX PRO 6000 family

This family records the September 4–5 decision funnel on one or two 96 GB RTX PRO 6000 Blackwell Max-Q cards under Docker Desktop/WSL2. The selected one-card SGLang arm uses Inferact NVFP4, DFlash2 K12, 1,024-token prefill chunks, FP8 target KV, BF16 Mamba state, ordinary extra_buffer, 96 state slots, 262,144 configured tokens, and C8. The matched TP2 arm is retained as a rejected correctness and topology control. RadixArk K8 is the lower-TTFT tradeoff; kelnei/vLLM MTP2 is an alternate-runtime gain over its exact no-spec control.

Use the measured recipe cards for values read from each native JSON artifact, or the full finding for the optimization funnel, workload boundary, correctness failure, and restoration proof. None of these recipes is promoted.

Inferact SGLang optimization family — finalist and rejected TP2 control

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K8-Compile"
status = "qualification-candidate"
source = "2026-09-04 matched compile arm for the current SGLang Qwen3.8 PRO 6000 optimization campaign"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft, torch.compile"
context_tokens = 262144
port = 39266
startup_timeout_seconds = 3600
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k8-compile"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k8-compile",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 2048", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 8",
  "--enable-torch-compile", "--torch-compile-max-bs 4",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39266",
]
note = "Changes only torch.compile from the measured K8/2K control. Retain cold compile time, capture failures, and warm performance. The compiled batch ceiling is four, so C8 includes the runtime's actual compiled/eager behavior. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "decode", "concurrency", "speculative-decoding", "compile-experiment"]
rationale = "Tests whether the Helix compile uplift survives this exact current runtime and local card."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B", "https://github.com/sgl-project/sglang/commit/5f55db35e926d50676f75b812640ea2410b0fe0e"]
community = ["https://huggingface.co/Inferact/Qwen3.8-27B-NVFP4/tree/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462", "https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-dflash2-matrix-recipes.toml", "docs/findings/2026-09-04-qwen38-27b-pro6000-possibility-plan.md"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K4"
status = "qualification-candidate"
source = "2026-09-04 matched DFlash2 draft-depth arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K4"
context_tokens = 262144
port = 39267
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k4"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k4",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 2048", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 4",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39267",
]
note = "Changes only DFlash2 draft depth from K8 to K4. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "decode", "concurrency", "speculative-decoding"]
rationale = "Tests whether lower verification overhead beats K8 at C1-C8."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B", "https://huggingface.co/incoai/Qwen3.8-27B-DFlash2/tree/dedf8df68adfb1afeaf7b7480c0a0243108177b4"]
community = ["https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-dflash2-matrix-recipes.toml"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K12"
status = "qualification-candidate"
source = "2026-09-04 matched DFlash2 draft-depth arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K12"
context_tokens = 262144
port = 39268
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 2048", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 12",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39268",
]
note = "Changes only DFlash2 draft depth from K8 to K12. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "decode", "concurrency", "speculative-decoding"]
rationale = "Tests whether longer accepted runs beat K8 after verification overhead."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B", "https://huggingface.co/incoai/Qwen3.8-27B-DFlash2/tree/dedf8df68adfb1afeaf7b7480c0a0243108177b4"]
community = ["https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-dflash2-matrix-recipes.toml"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K16"
status = "qualification-candidate"
source = "2026-09-04 matched DFlash2 draft-depth arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K16"
context_tokens = 262144
port = 39269
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k16"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k16",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 2048", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 16",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39269",
]
note = "Changes only DFlash2 draft depth from K8 to K16. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "decode", "concurrency", "speculative-decoding"]
rationale = "Tests the upper draft-depth lead without extrapolating from K8 acceptance."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B", "https://huggingface.co/incoai/Qwen3.8-27B-DFlash2/tree/dedf8df68adfb1afeaf7b7480c0a0243108177b4"]
community = ["https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-dflash2-matrix-recipes.toml"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K8-Chunk1K"
status = "qualification-candidate"
source = "2026-09-04 matched chunked-prefill arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K8"
context_tokens = 262144
port = 39270
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k8-chunk1k"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k8-chunk1k",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 1024", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 8",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39270",
]
note = "Changes only chunked prefill from 2K to 1K on DFlash2 K8. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "prefill", "concurrency", "long-context", "speculative-decoding"]
rationale = "Tests whether smaller chunks improve mixed short/long responsiveness."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B"]
community = ["https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-dflash2-matrix-recipes.toml"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-TP2-DFlash2-K12-Chunk1K"
status = "qualification-candidate"
source = "2026-09-04 matched TP=2 topology arm for the TP=1 K12/1K finalist"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 192
vram_note = "Two separate PCIe devices without NVLink under WSL2; aggregate TP-sharded capacity, never unified memory."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = [
  "bash", "-lc",
  '''set -e; LP=/sgl-workspace/sglang/python/sglang/srt/layers/logits_processor.py; ACTUAL="$(sha256sum "$LP" | awk '{print $1}')"; case "$ACTUAL" in 43a0674df59067223fb793d78bb9f947049d2f983704a09a3c67282bf654cad0) test "$(grep -Fc 'enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,' "$LP")" = 1 || { echo 'anvil patch: logits gatherer anchor drift' >&2; exit 3; }; sed -i 's/enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,/enabled=False,/' "$LP" ;; 83986d97cd5741515477cc182a4c72d1fd6f832f26f97ffa94093740db149627) ;; *) echo "anvil patch: unknown logits processor hash $ACTUAL" >&2; exit 4 ;; esac; echo '83986d97cd5741515477cc182a4c72d1fd6f832f26f97ffa94093740db149627  /sgl-workspace/sglang/python/sglang/srt/layers/logits_processor.py' | sha256sum -c -; test "$(grep -Fc 'enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,' "$LP")" = 0 || { echo 'anvil patch: old logits gatherer remains' >&2; exit 5; }; echo 'anvil patch: TP2 logits gatherer disabled'; exec python3 -m sglang.launch_server "$@"''',
  "qwen38-tp2",
]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K12"
context_tokens = 262144
port = 39277
startup_timeout_seconds = 3000
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-tp2-dflash2-k12-chunk1k"
ipc = "host"
shm_size = "64gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0",
  "SGLANG_USE_CUDA_IPC_TRANSPORT=0", "NCCL_CUMEM_ENABLE=0", "NCCL_DEBUG=INFO",
]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-tp2-dflash2-k12-chunk1k",
  "--tp-size 2", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--disable-custom-all-reduce",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 1024", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 12",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39277",
]
note = "Changes topology only from the TP=1 K12/1K finalist. Uses the locally qualified conservative WSL2 NCCL controls plus a checksum-gated, idempotent logits-gather patch after the pristine image crashed in symmetric-memory rendezvous. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "prefill", "concurrency", "long-context", "speculative-decoding", "tensor-parallel"]
rationale = "Tests whether TP=2 long-prefill and concurrent-context gains offset PCIe collectives and DFlash candidate all-gathers."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B", "https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/config.json"]
community = ["https://github.com/sgl-project/sglang/issues/15181", "https://forums.developer.nvidia.com/t/qwen3-8-27b-on-dual-sparks/380350"]
local = ["docs/findings/2026-08-14-qwen38-27b-tp-mtp-context-matrix.md", "configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-optimization-recipes.toml"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K12-Chunk1K-Lazy96"
status = "qualification-candidate"
source = "2026-09-04 winning-depth Mamba strategy arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K12"
context_tokens = 262144
port = 39275
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-chunk1k-lazy96"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-chunk1k-lazy96",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 1024", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer_lazy", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 12",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39275",
]
note = "Changes only Mamba radix strategy from extra_buffer to extra_buffer_lazy on the K12/1K leader, retaining 96 slots. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "prefill", "concurrency", "long-context", "speculative-decoding", "mamba-state"]
rationale = "Tests the official higher-throughput lazy state strategy without confounding it with a smaller slot cap."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B"]
community = ["https://helix.ml/blog/the-ceiling-was-a-state-cache", "https://github.com/sgl-project/sglang/issues/36935"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-optimization-recipes.toml"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K12-Chunk1K-Mamba40"
status = "qualification-candidate"
source = "2026-09-04 winning-depth minimum-active-state arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K12"
context_tokens = 262144
port = 39276
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-chunk1k-mamba40"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-chunk1k-mamba40",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 1024", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 40",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 12",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39276",
]
note = "Changes only the Mamba slot cap from 96 to the official C8 extra_buffer active minimum of 40. Divergent long-prefix cache reuse remains a mandatory gate. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "prefill", "concurrency", "long-context", "speculative-decoding", "mamba-state"]
rationale = "Measures reclaimed KV capacity and any branch-cache penalty from the minimum active-state allocation."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B"]
community = ["https://helix.ml/blog/the-ceiling-was-a-state-cache", "https://github.com/sgl-project/sglang/issues/36935"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-optimization-recipes.toml"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K12-Chunk1K"
status = "qualification-candidate"
source = "2026-09-04 winning-depth chunked-prefill combination arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K12"
context_tokens = 262144
port = 39272
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-chunk1k"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-chunk1k",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 1024", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 12",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39272",
]
note = "Changes only chunked prefill from 2K to 1K on the measured K12 winner. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "prefill", "concurrency", "long-context", "speculative-decoding"]
rationale = "Tests smaller prefill chunks while retaining the measured winning DFlash2 depth."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B"]
community = ["https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-optimization-recipes.toml"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K12-Chunk8K"
status = "qualification-candidate"
source = "2026-09-04 winning-depth chunked-prefill combination arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K12"
context_tokens = 262144
port = 39273
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-chunk8k"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-chunk8k",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 8192", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 12",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39273",
]
note = "Changes only chunked prefill from 2K to 8K on the measured K12 winner. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "prefill", "concurrency", "long-context", "speculative-decoding"]
rationale = "Tests larger prefill chunks while retaining the measured winning DFlash2 depth."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B"]
community = ["https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-optimization-recipes.toml"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K12-Compile"
status = "qualification-candidate"
source = "2026-09-04 winning-depth torch.compile combination arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K12, torch.compile"
context_tokens = 262144
port = 39274
startup_timeout_seconds = 3600
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-compile"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k12-compile",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 2048", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 12",
  "--enable-torch-compile", "--torch-compile-max-bs 8",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39274",
]
note = "Changes only torch.compile from the measured K12/2K winner. Retain cold compile time, capture failures, and warm performance. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "decode", "concurrency", "speculative-decoding", "compile-experiment"]
rationale = "Tests the Helix compile lever while retaining the measured winning DFlash2 depth."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B", "https://github.com/sgl-project/sglang/commit/5f55db35e926d50676f75b812640ea2410b0fe0e"]
community = ["https://huggingface.co/Inferact/Qwen3.8-27B-NVFP4/tree/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462", "https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-optimization-recipes.toml", "docs/findings/2026-09-04-qwen38-27b-pro6000-possibility-plan.md"]

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DFlash2-K8-Chunk8K"
status = "qualification-candidate"
source = "2026-09-04 matched chunked-prefill arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during one bounded campaign interruption."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K8"
context_tokens = 262144
port = 39271
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dflash2-k8-chunk8k"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dflash2-k8-chunk8k",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 8192", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 8",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39271",
]
note = "Changes only chunked prefill from 2K to 8K on DFlash2 K8. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "prefill", "concurrency", "long-context", "speculative-decoding"]
rationale = "Tests whether larger chunks improve long-context prefill without unacceptable short-request latency."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B"]
community = ["https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = ["configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-dflash2-matrix-recipes.toml"]

RadixArk SGLang K8 lower-TTFT tradeoff

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "RadixArk/Qwen3.8-27B-NVFP4-PRO6000-C8-DFlash2"
status = "qualification-candidate"
source = "2026-09-04 current SGLang Qwen3.8 DFlash2 image translated into a matched RTX PRO 6000 K8 arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "RadixArk/Qwen3.8-27B-NVFP4"
revision = "319f741cce68d7914884900c138a1fbb70a42f30"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "One card in a dedicated TP=1 qualification lane; the paired card is tested separately before any TP=2 comparison."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--RadixArk--Qwen3.8-27B-NVFP4/snapshots/319f741cce68d7914884900c138a1fbb70a42f30"
quantization = "ModelOpt NVFP4 W4A4 target including the restored quantized LM head, FP8 E4M3 target KV, and BF16 incoai DFlash2 draft"
context_tokens = 262144
port = 39261
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-radixark-nvfp4-sglang-pro6000-c8-dflash2"
ipc = "host"
shm_size = "32gb"
named_volumes = [
  "qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang",
]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTHONHASHSEED=0",
  "SGLANG_USE_CUDA_IPC_TRANSPORT=0",
]
flags = [
  "--served-model-name qwen38-27b-radixark-nvfp4-sglang-pro6000-c8-dflash2",
  "--tp-size 1",
  "--language-only",
  "--context-length 262144",
  "--max-running-requests 8",
  "--cuda-graph-max-bs 8",
  "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3",
  "--attention-backend flashinfer",
  "--chunked-prefill-size 2048",
  "--disable-prefill-cuda-graph",
  "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer",
  "--mamba-ssm-dtype bfloat16",
  "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3",
  "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant",
  "--speculative-num-draft-tokens 8",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'",
  "--enable-metrics",
  "--host 0.0.0.0",
  "--port 39261",
]
note = "Matched DFlash2 K8 arm for the 2026-09-04 RTX PRO 6000 possibility campaign. It differs from the no-spec control only where DFlash2 requires a BF16 draft checkpoint, K8 verification, and additional Mamba state slots. The target revision restores the quantized LM head after vLLM PR 52816 and corresponding SGLang work removed the earlier dense-head restriction that shaped the Helix experiment. The 96-slot allocation follows the current C8 DFlash2 community recipe and is verified at startup rather than treated as proof of eight-way admission. Prefix-cache repeats, mixed simultaneous prompts, structured output, tools, and request isolation are hard gates because current issue reports describe cache regressions and potential cross-request state corruption. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "decode", "concurrency", "long-context", "text", "coding", "agentic", "speculative-decoding"]
rationale = "Tests the most plausible current path to high single-stream and aggregate throughput on one 96 GB SM120 card without treating a best-case external cell as a local result."

[recipe.sources]
official = [
  "https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B",
  "https://github.com/sgl-project/sglang/commit/5f55db35e926d50676f75b812640ea2410b0fe0e",
  "https://github.com/vllm-project/vllm/pull/52816",
  "https://huggingface.co/incoai/Qwen3.8-27B-DFlash2/tree/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
]
community = [
  "https://huggingface.co/RadixArk/Qwen3.8-27B-NVFP4/tree/319f741cce68d7914884900c138a1fbb70a42f30",
  "https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000",
  "https://github.com/sgl-project/sglang/issues/36548",
]
local = [
  "configs/qwen38-27b-radixark-nvfp4-sglang-pro6000-c8-nospec-recipe.toml",
  "docs/benchmarks/models/qwen38-27b.md",
  "docs/findings/2026-09-04-qwen38-27b-pro6000-possibility-plan.md",
]

kelnei vLLM MTP2

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "kelnei/Qwen3.8-27B-NVFP4-vLLM0271-MTP2-PRO6000"
status = "qualification-candidate"
source = "2026-09-05 current RTX PRO 6000 model-card reproduction and alternate-runtime challenger"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "kelnei/Qwen3.8-27B-NVFP4"
revision = "29099dc7004e5731173af5c5fb5253466aee219c"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during the bounded Qwen3.8 optimization campaign."

[recipe.serve]
engine = "vllm"
image = "vllm/vllm-openai:v0.27.1@sha256:c2f3b1b964e47809b722b5e75b61b1e7b39a50f70388cf2bf2418f16a9f31da2"
entrypoint = ["python3", "-m", "vllm.entrypoints.openai.api_server"]
model_flag = "--model"
model_path = "/root/.cache/huggingface/hub/models--kelnei--Qwen3.8-27B-NVFP4/snapshots/29099dc7004e5731173af5c5fb5253466aee219c"
quantization = "compressed-tensors NVFP4 W4A4 with FP8 attention, DeltaNet, late MLP layers, lm_head, and KV; integrated MTP head"
context_tokens = 262144
port = 39279
health = "/health"
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-kelnei-nvfp4-vllm0271-mtp2"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-kelnei-vllm-0271:/root/.cache/vllm"]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTHONHASHSEED=0",
  "VLLM_WSL2_ENABLE_PIN_MEMORY=1",
  "FLASHINFER_CUDA_ARCH_LIST=12.0f",
  "MAX_JOBS=2",
]
flags = [
  "--served-model-name qwen38-27b-kelnei-nvfp4-vllm0271-mtp2",
  "--language-model-only",
  "--tensor-parallel-size 1",
  "--max-model-len 262144",
  "--max-num-seqs 8",
  "--gpu-memory-utilization 0.90",
  "--kv-cache-dtype fp8",
  "--no-enable-prefix-caching",
  "--speculative-config '{\"method\":\"mtp\",\"num_speculative_tokens\":2}'",
  "--reasoning-parser qwen3",
  "--enable-auto-tool-choice",
  "--tool-call-parser qwen3_coder",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'",
  "--generation-config vllm",
  "--host 0.0.0.0",
  "--port 39279",
]
note = "Exact current publisher checkpoint and same-hardware vLLM 0.27.1 MTP2 prior. Startup must prove an optimized Cutlass NVFP4 kernel and active MTP counters. Prefix caching stays off so the throughput comparison is not warmed by shared-prefix reuse. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "alternate-runtime", "latency", "decode", "concurrency", "speculative-decoding"]
not_suited = ["automatic-promotion", "SGLang", "DFlash2"]
rationale = "Tests the strongest current same-card external throughput prior against the measured SGLang finalists without inheriting publisher results as local evidence."

[recipe.sources]
official = [
  "https://huggingface.co/Qwen/Qwen3.8-27B",
  "https://recipes.vllm.ai/Qwen/Qwen3.8-27B",
  "https://docs.vllm.ai/en/v0.27.1/",
]
community = [
  "https://huggingface.co/kelnei/Qwen3.8-27B-NVFP4/tree/29099dc7004e5731173af5c5fb5253466aee219c",
  "https://github.com/kelnei/nvfp4-vllm",
]
local = [
  "docs/findings/2026-09-04-qwen38-27b-pro6000-possibility-plan.md",
]

kelnei vLLM matched no-speculation control

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "kelnei/Qwen3.8-27B-NVFP4-vLLM0271-NoSpec-PRO6000"
status = "qualification-control"
source = "2026-09-05 matched no-spec control for the RTX PRO 6000 kelnei MTP2 challenger"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "kelnei/Qwen3.8-27B-NVFP4"
revision = "29099dc7004e5731173af5c5fb5253466aee219c"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Dedicated TP=1 qualification lane during the bounded Qwen3.8 optimization campaign."

[recipe.serve]
engine = "vllm"
image = "vllm/vllm-openai:v0.27.1@sha256:c2f3b1b964e47809b722b5e75b61b1e7b39a50f70388cf2bf2418f16a9f31da2"
entrypoint = ["python3", "-m", "vllm.entrypoints.openai.api_server"]
model_flag = "--model"
model_path = "/root/.cache/huggingface/hub/models--kelnei--Qwen3.8-27B-NVFP4/snapshots/29099dc7004e5731173af5c5fb5253466aee219c"
quantization = "compressed-tensors NVFP4 W4A4 with FP8 attention, DeltaNet, late MLP layers, lm_head, and KV; MTP disabled"
context_tokens = 262144
port = 39280
health = "/health"
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-kelnei-nvfp4-vllm0271-nospec"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-kelnei-vllm-0271:/root/.cache/vllm"]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTHONHASHSEED=0",
  "VLLM_WSL2_ENABLE_PIN_MEMORY=1",
  "FLASHINFER_CUDA_ARCH_LIST=12.0f",
  "MAX_JOBS=2",
]
flags = [
  "--served-model-name qwen38-27b-kelnei-nvfp4-vllm0271-nospec",
  "--language-model-only",
  "--tensor-parallel-size 1",
  "--max-model-len 262144",
  "--max-num-seqs 8",
  "--gpu-memory-utilization 0.90",
  "--kv-cache-dtype fp8",
  "--no-enable-prefix-caching",
  "--reasoning-parser qwen3",
  "--enable-auto-tool-choice",
  "--tool-call-parser qwen3_coder",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'",
  "--generation-config vllm",
  "--host 0.0.0.0",
  "--port 39280",
]
note = "Changes only speculative decoding from the paired MTP2 arm. The same exact checkpoint, runtime, memory, context, concurrency, parser, and cache controls are retained. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification-control", "alternate-runtime", "latency", "decode", "concurrency"]
not_suited = ["automatic-promotion", "SGLang", "DFlash2"]
rationale = "Measures whether MTP2 provides a local end-to-end benefit instead of inheriting the publisher's external comparison."

[recipe.sources]
official = [
  "https://huggingface.co/Qwen/Qwen3.8-27B",
  "https://recipes.vllm.ai/Qwen/Qwen3.8-27B",
  "https://docs.vllm.ai/en/v0.27.1/",
]
community = [
  "https://huggingface.co/kelnei/Qwen3.8-27B-NVFP4/tree/29099dc7004e5731173af5c5fb5253466aee219c",
  "https://github.com/kelnei/nvfp4-vllm",
]
local = [
  "configs/qwen38-27b-kelnei-nvfp4-vllm0271-pro6000-mtp2-recipe.toml",
]

Second independent TP1 replica for the measured DP2 topology

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "Inferact/Qwen3.8-27B-NVFP4-PRO6000-DP2-ReplicaB-DFlash2-K12-Chunk1K"
status = "qualification-candidate"
source = "2026-09-05 second independent TP=1 replica for a matched two-card data-parallel throughput measurement"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Inferact/Qwen3.8-27B-NVFP4"
revision = "6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "Replica B occupies one dedicated card; aggregate DP=2 evidence is valid only when paired with the identical pinned replica A on the other card."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Inferact--Qwen3.8-27B-NVFP4/snapshots/6128240ebaf4eaa7bad2b3d1c72c37d677c5f462"
quantization = "ModelOpt NVFP4 W4A4 target, FP8 E4M3 target KV, BF16 DFlash2 draft K12"
context_tokens = 262144
port = 39278
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-inferact-nvfp4-pro6000-dp2-replica-b-dflash2-k12-chunk1k"
ipc = "host"
shm_size = "32gb"
named_volumes = ["qwen38-pro6000-dflash2-sglang-cache:/root/.cache/sglang"]
env = ["CUDA_DEVICE_ORDER=PCI_BUS_ID", "PYTHONHASHSEED=0", "SGLANG_USE_CUDA_IPC_TRANSPORT=0"]
flags = [
  "--served-model-name qwen38-27b-inferact-nvfp4-pro6000-dp2-replica-b-dflash2-k12-chunk1k",
  "--tp-size 1", "--language-only", "--context-length 262144",
  "--max-running-requests 8", "--cuda-graph-max-bs 8", "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3", "--attention-backend flashinfer",
  "--chunked-prefill-size 1024", "--disable-prefill-cuda-graph", "--disable-flashinfer-autotune",
  "--mamba-radix-cache-strategy extra_buffer", "--mamba-ssm-dtype bfloat16", "--max-mamba-cache-size 96",
  "--reasoning-parser qwen3", "--tool-call-parser qwen3_coder",
  "--speculative-algorithm DFLASH",
  "--speculative-draft-model-path /root/.cache/huggingface/hub/models--incoai--Qwen3.8-27B-DFlash2/snapshots/dedf8df68adfb1afeaf7b7480c0a0243108177b4",
  "--speculative-draft-model-quantization unquant", "--speculative-num-draft-tokens 12",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'", "--enable-metrics",
  "--host 0.0.0.0", "--port 39278",
]
note = "Exact second replica of the measured Inferact K12/1K finalist with only its served name and loopback port changed. It enables measured DP=2 without TP collectives. Direct-only; no route or promotion authority."

[recipe.fit]
suited = ["qualification", "latency", "decode", "concurrency", "data-parallel", "aggregate-throughput"]
rationale = "Measures whether two independent TP=1 replicas scale aggregate decode throughput better than the failed TP=2 topology on two PCIe-attached GPUs under WSL2."

[recipe.sources]
official = ["https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B"]
community = ["https://helix.ml/blog/chasing-454-toks-qwen38-rtx-pro-6000"]
local = [
  "configs/qwen38-27b-inferact-nvfp4-sglang-pro6000-optimization-recipes.toml",
  "docs/findings/2026-09-04-qwen38-27b-pro6000-possibility-plan.md",
]

GLM-5.3-Flash 524K DFlash2 K5 and control

The selected arm and its no-speculation control pin target revision 319d66a8b53092b491f698440ecea781e4ddd4e4, the same local image digest, TP=2/DCP=2, FP8 DS-MLA target KV, 524,288-token context, batch-2,048 scheduler, max-seq-16 admission, WSL2/NCCL controls, parsers, and image/OCR envelope. The candidate additionally loads DFlash2 revision dc77ff1c99eeb2df044ee3d4f0094eb033fee410 with five draft tokens and BF16 draft KV. Both use vllm-hfcache for model snapshots and glm53-purtell-k3-dflash2-vllm-cache for /root/.cache.

Result and evidence · Model dossier

DFlash2 K5 — full tracked recipe

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3-v1"
status = "qualification-candidate"
source = "Matched 524K DFlash2 K5 candidate on the hash-gated Anvil xgrammar-corrected Purtell runtime"
family = "glm-5.3-flash-multimodal-moe"

[recipe.download]
repo = "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3-v1"
revision = "319d66a8b53092b491f698440ecea781e4ddd4e4"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 192
vram_note = "Aggregate PCIe VRAM without NVLink under WSL2/Docker Desktop; exclusive TP=2/DCP=2 qualification lane."

[recipe.serve]
engine = "vllm-glm53-exl3-k3-b12x-dflash2-xgrammar-fixed"
allow_cuda_visible_devices_index = true
image = "anvil-glm53-xgrammar@sha256:4909e318ba1348a179824e210f90c268d6fc68e8b4e514af4782e26e6a1e5939"
model_path = "/root/.cache/huggingface/hub/models--wrldsuksgo2mars--GLM-5.3-Flash-EXL3-K3-v1/snapshots/319d66a8b53092b491f698440ecea781e4ddd4e4"
named_volumes = [
  "glm53-purtell-k3-dflash2-vllm-cache:/root/.cache",
]
quantization = "EXL3/MCG K3 routed experts with native-precision attention, dense/shared tensors, vision, and MTP; FP8 DS-MLA target KV; BF16 DFlash2 K5 draft KV"
context_tokens = 524288
port = 8001
startup_timeout_seconds = 3600
served_model_name = "glm53-flash-exl3-k3-dflash2-k5-fp8-tp2-524k-vision-xgfix"
ipc = "host"
shm_size = "32gb"
env = [
  "HF_HUB_OFFLINE=1",
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False",
  "NCCL_CUMEM_ENABLE=0",
  "NCCL_CUMEM_HOST_ENABLE=0",
  "NCCL_DEBUG=WARN",
  "VLLM_USE_V2_MODEL_RUNNER=1",
  "VLLM_WSL2_ENABLE_PIN_MEMORY=1",
  "VLLM_ENGINE_READY_TIMEOUT_S=3600",
  "VLLM_ADAPTIVE_MTP=0",
  "VLLM_ENABLE_PCIE_ALLREDUCE=0",
  "VLLM_PCIE_ALLREDUCE_BACKEND=b12x",
  "VLLM_PCIE_ONESHOT_ALLREDUCE_MAX_SIZE=384KB",
  "VLLM_B12X_PCIE_EAGER=0",
  "VLLM_B12X_DCP_A2A=0",
  "VLLM_USE_B12X_SPARSE_INDEXER=1",
  "VLLM_USE_B12X_KPOOL_INDEXER=1",
  "VLLM_DCP_GLOBAL_TOPK=1",
  "VLLM_DCP_QUERY_SPLIT=0",
  "VLLM_DCP_TOPK_OWNER_MERGE=1",
  "VLLM_B12X_DCP_TOPK_OWNER_EXCHANGE=0",
  "VLLM_B12X_DCP_TOPK_MIN_ROWS=128",
  "VLLM_B12X_DCP_TOPK_MAX_ROWS=2048",
  "KV_FP8_ROPE=0",
  "VLLM_NVFP4_MLA_DYNAMIC_SCALE=0",
  "VLLM_EXL3_TRELLIS_MIN_M=1",
  "VLLM_EXL3_TRELLIS_MAX_M=32",
  "VLLM_EXL3_TRELLIS_BLOCK_M=8",
  "VLLM_EXL3_PREFILL_TRELLIS=1",
  "VLLM_EXL3_PREFILL_BLOCK_M=64",
  "VLLM_EXL3_PREFILL_CAPACITY=1024",
  "B12X_EXL3_BF16_EPILOGUE=1",
  "B12X_EXL3_BF16_GEMV=1",
  "VLLM_B12X_GLM_H64_QUERY_PROJ=auto",
  "VLLM_USE_B12X_MHC=auto",
]
flags = [
  "--served-model-name glm53-flash-exl3-k3-dflash2-k5-fp8-tp2-524k-vision-xgfix",
  "--host 0.0.0.0",
  "--port 8001",
  "--tensor-parallel-size 2",
  "--decode-context-parallel-size 2",
  "--dcp-comm-backend ag_rs",
  "--disable-custom-all-reduce",
  "--speculative-config '{\"method\":\"dflash\",\"model\":\"/root/.cache/huggingface/hub/models--incoai--GLM-5.3-Flash-DFlash2/snapshots/dc77ff1c99eeb2df044ee3d4f0094eb033fee410\",\"num_speculative_tokens\":5,\"kv_cache_dtype\":\"bfloat16\"}'",
  "--limit-mm-per-prompt '{\"image\":16,\"video\":0}'",
  "--enable-prefix-caching",
  "--mamba-cache-mode align",
  "--attention-backend B12X_MLA_SPARSE",
  "--max-model-len 524288",
  "--max-num-batched-tokens 2048",
  "--max-num-seqs 16",
  "--max-cudagraph-capture-size 96",
  "--gpu-memory-utilization 0.950",
  "--kv-cache-dtype fp8_ds_mla",
  "--no-enable-flashinfer-autotune",
  "--enable-auto-tool-choice",
  "--tool-call-parser glm47",
  "--reasoning-parser glm45",
]
note = "Matched DFlash2 K5 candidate for the corrected-runtime qualification. It keeps the exact K3 target, image, context, TP/DCP, KV dtype, scheduler, vision, tool, parser, WSL2, and memory settings used by the no-spec control; only served identity and speculative-config differ. Qualification must exceed the current 67.9 output tok/s near the 240K lane, pass the 250K-class prompt plus output reserve at C2, and produce repeated structured output without Failed-to-advance-FSM errors. DFlash2 is CC-BY-NC-ND-4.0 and remains limited to evaluation/noncommercial use without separate permission."

[recipe.fit]
suited = ["qualification", "coding-agent", "tool-use", "long-context", "speculative-decoding", "high-concurrency", "text", "image", "ocr"]
rationale = "Tests whether corrected DFlash2 structured generation preserves the current speed advantage while halving configured context to a higher-headroom 524K deployment envelope."

[recipe.sources]
official = [
  "https://huggingface.co/wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3-v1/tree/319d66a8b53092b491f698440ecea781e4ddd4e4",
  "https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2/tree/dc77ff1c99eeb2df044ee3d4f0094eb033fee410",
  "https://github.com/vllm-project/vllm/commit/12f64b39d29282437e35be9aa5db432fb2a1a6e6",
  "https://github.com/vllm-project/vllm/commit/c6e19b3be24338759a443e03c8325d76da9ee202",
]
community = [
  "https://github.com/tpurtell/glm-5.3-flash-ext3-4-bit-2x-rtx/tree/d46fdeddf8c6fec2d4595b65535a32d80a5af787",
]
local = [
  "configs/runtime-patches/vllm/487ecf187-xgrammar-spec-reasoning-end/Dockerfile",
  "configs/runtime-patches/vllm/487ecf187-xgrammar-spec-reasoning-end/xgrammar-spec-reasoning-end.patch",
  "configs/glm53-flash-purtell-k3-nospec-fp8-524k-vision-xgrammar-sm120-tp2-wsl2-recipe.toml",
  "docs/findings/2026-08-31-glm53-xgrammar-524k-qualification-evidence/feasibility-result.md",
]

No speculation — full tracked control

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3-v1"
status = "qualification-candidate"
source = "Matched 524K no-spec control on the hash-gated Anvil xgrammar-corrected Purtell runtime"
family = "glm-5.3-flash-multimodal-moe"

[recipe.download]
repo = "wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3-v1"
revision = "319d66a8b53092b491f698440ecea781e4ddd4e4"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 192
vram_note = "Aggregate PCIe VRAM without NVLink under WSL2/Docker Desktop; exclusive TP=2/DCP=2 qualification lane."

[recipe.serve]
engine = "vllm-glm53-exl3-k3-b12x-xgrammar-fixed"
allow_cuda_visible_devices_index = true
image = "anvil-glm53-xgrammar@sha256:4909e318ba1348a179824e210f90c268d6fc68e8b4e514af4782e26e6a1e5939"
model_path = "/root/.cache/huggingface/hub/models--wrldsuksgo2mars--GLM-5.3-Flash-EXL3-K3-v1/snapshots/319d66a8b53092b491f698440ecea781e4ddd4e4"
named_volumes = [
  "glm53-purtell-k3-dflash2-vllm-cache:/root/.cache",
]
quantization = "EXL3/MCG K3 routed experts with native-precision attention, dense/shared tensors, vision, and MTP; FP8 DS-MLA target KV; no speculative draft"
context_tokens = 524288
port = 8001
startup_timeout_seconds = 3600
served_model_name = "glm53-flash-exl3-k3-nospec-fp8-tp2-524k-vision-xgfix"
ipc = "host"
shm_size = "32gb"
env = [
  "HF_HUB_OFFLINE=1",
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False",
  "NCCL_CUMEM_ENABLE=0",
  "NCCL_CUMEM_HOST_ENABLE=0",
  "NCCL_DEBUG=WARN",
  "VLLM_USE_V2_MODEL_RUNNER=1",
  "VLLM_WSL2_ENABLE_PIN_MEMORY=1",
  "VLLM_ENGINE_READY_TIMEOUT_S=3600",
  "VLLM_ADAPTIVE_MTP=0",
  "VLLM_ENABLE_PCIE_ALLREDUCE=0",
  "VLLM_PCIE_ALLREDUCE_BACKEND=b12x",
  "VLLM_PCIE_ONESHOT_ALLREDUCE_MAX_SIZE=384KB",
  "VLLM_B12X_PCIE_EAGER=0",
  "VLLM_B12X_DCP_A2A=0",
  "VLLM_USE_B12X_SPARSE_INDEXER=1",
  "VLLM_USE_B12X_KPOOL_INDEXER=1",
  "VLLM_DCP_GLOBAL_TOPK=1",
  "VLLM_DCP_QUERY_SPLIT=0",
  "VLLM_DCP_TOPK_OWNER_MERGE=1",
  "VLLM_B12X_DCP_TOPK_OWNER_EXCHANGE=0",
  "VLLM_B12X_DCP_TOPK_MIN_ROWS=128",
  "VLLM_B12X_DCP_TOPK_MAX_ROWS=2048",
  "KV_FP8_ROPE=0",
  "VLLM_NVFP4_MLA_DYNAMIC_SCALE=0",
  "VLLM_EXL3_TRELLIS_MIN_M=1",
  "VLLM_EXL3_TRELLIS_MAX_M=32",
  "VLLM_EXL3_TRELLIS_BLOCK_M=8",
  "VLLM_EXL3_PREFILL_TRELLIS=1",
  "VLLM_EXL3_PREFILL_BLOCK_M=64",
  "VLLM_EXL3_PREFILL_CAPACITY=1024",
  "B12X_EXL3_BF16_EPILOGUE=1",
  "B12X_EXL3_BF16_GEMV=1",
  "VLLM_B12X_GLM_H64_QUERY_PROJ=auto",
  "VLLM_USE_B12X_MHC=auto",
]
flags = [
  "--served-model-name glm53-flash-exl3-k3-nospec-fp8-tp2-524k-vision-xgfix",
  "--host 0.0.0.0",
  "--port 8001",
  "--tensor-parallel-size 2",
  "--decode-context-parallel-size 2",
  "--dcp-comm-backend ag_rs",
  "--disable-custom-all-reduce",
  "--limit-mm-per-prompt '{\"image\":16,\"video\":0}'",
  "--enable-prefix-caching",
  "--mamba-cache-mode align",
  "--attention-backend B12X_MLA_SPARSE",
  "--max-model-len 524288",
  "--max-num-batched-tokens 2048",
  "--max-num-seqs 16",
  "--max-cudagraph-capture-size 96",
  "--gpu-memory-utilization 0.950",
  "--kv-cache-dtype fp8_ds_mla",
  "--no-enable-flashinfer-autotune",
  "--enable-auto-tool-choice",
  "--tool-call-parser glm47",
  "--reasoning-parser glm45",
]
note = "Matched no-spec control for the corrected-runtime qualification. It keeps the exact K3 target, image, context, TP/DCP, KV dtype, scheduler, vision, tool, parser, WSL2, and memory settings used by the DFlash2 arm; only served identity and the absence of speculative-config differ. Qualification must prove a 250K-class prompt plus output reserve at C2 and establish the matched throughput control."

[recipe.fit]
suited = ["qualification", "coding-agent", "tool-use", "long-context", "high-concurrency", "text", "image", "ocr"]
rationale = "Provides the otherwise-matched 524K control required to attribute any measured gain to DFlash2 rather than context, batching, runtime, or parser drift."

[recipe.sources]
official = [
  "https://huggingface.co/wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K3-v1/tree/319d66a8b53092b491f698440ecea781e4ddd4e4",
  "https://github.com/vllm-project/vllm/commit/12f64b39d29282437e35be9aa5db432fb2a1a6e6",
  "https://github.com/vllm-project/vllm/commit/c6e19b3be24338759a443e03c8325d76da9ee202",
]
community = [
  "https://github.com/tpurtell/glm-5.3-flash-ext3-4-bit-2x-rtx/tree/d46fdeddf8c6fec2d4595b65535a32d80a5af787",
]
local = [
  "configs/runtime-patches/vllm/487ecf187-xgrammar-spec-reasoning-end/Dockerfile",
  "configs/runtime-patches/vllm/487ecf187-xgrammar-spec-reasoning-end/xgrammar-spec-reasoning-end.patch",
  "docs/findings/2026-08-31-glm53-xgrammar-524k-qualification-evidence/feasibility-result.md",
]

GLM-5.3-Flash SGLang SM120 adaptive MTP and control

This family pins ormandj checkpoint revision c3cbb9891b67c741bcbf6b176dd7af9265b069db and rc14 image digest 0c0637959c3931829f05154087bbefd2c50003fb9b2010200ce0ec82f4d71a53. The qualified WSL2 translation uses TP=2, FP8 KV, explicit thinking control, image/OCR, and hash-gated source/template patches. The 131K matched pair isolates adaptive EAGLE. The human-approved 393,216/C1 adaptive profile is the published current contract after an explicit model-only reserve waiver; the 245,760/C1 profile remains the conservative verified fallback. The 499K/C4 profile is rejected and the 499K/C1 profile remains unverified.

Qualification evidence · Promotion · Model dossier

Adaptive MTP, 393,216 tokens — current tracked recipe

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO"
status = "verified"
source = "2026-09-02 fully qualified 393216-token C1 profile, reclassified after the operator waived the default 3 GiB reserve for exclusive model-only GPUs"
family = "glm-5.3-flash-multimodal-moe"
total_params_b = 744
active_params_b = 40

[recipe.download]
repo = "ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO"
revision = "c3cbb9891b67c741bcbf6b176dd7af9265b069db"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 192
vram_note = "Aggregate PCIe VRAM without NVLink under WSL2; exclusive TP=2 only. The default 3072 MiB/card campaign reserve is explicitly waived for these model-only devices. The complete workload left 2101 MiB free per card with no OOM, CUDA error, crash, restart, corruption, or unexplained request loss."

[recipe.serve]
engine = "sglang-glm53-sm120-v0.1.1-rc14"
allow_cuda_visible_devices_index = true
image = "ghcr.io/ormandj/sglang-glm53-flash-sm120@sha256:0c0637959c3931829f05154087bbefd2c50003fb9b2010200ce0ec82f4d71a53"
entrypoint = [
  "bash",
  "-lc",
  '''set -e; printf '%s\n' '{"1":{"candidate_steps":[3,5],"up_hysteresis":0.0,"down_hysteresis":-0.25,"ceiling_coeff":0},"4":{"candidate_steps":[3,5],"up_hysteresis":0.0,"down_hysteresis":-0.25,"ceiling_coeff":0}}' > /tmp/glm53-adaptive.json; LP=/opt/sglang-source/python/sglang/srt/layers/logits_processor.py; echo '05cfcbd9c1439af751a22fb03490a2e5a252ea4bf5ad5febdae1eb87dce1e9a7  /opt/sglang-source/python/sglang/srt/layers/logits_processor.py' | sha256sum -c -; test "$(grep -Fc 'enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,' "$LP")" = 1 || { echo 'anvil patch: logits gatherer anchor drift' >&2; exit 3; }; sed -i 's/enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,/enabled=False,/' "$LP"; grep -Fq 'enabled=False,' "$LP" || { echo 'anvil patch: logits gatherer postcondition failed' >&2; exit 4; }; SRC=/root/.cache/huggingface/hub/models--ormandj--GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO/snapshots/c3cbb9891b67c741bcbf6b176dd7af9265b069db/chat_template.jinja; DST=/root/.cache/glm53-ormandj-enable-thinking-chat-template.jinja; python3 -c 'import base64,hashlib,os,pathlib,sys; src=pathlib.Path(sys.argv[1]); dst=pathlib.Path(sys.argv[2]); raw=src.read_bytes(); hashlib.sha256(raw).hexdigest()=="34d5ee66b12fa6446cdae131c352b8f68cd85369e0e6fda115583805fada3891" or sys.exit("anvil patch: source chat template hash drift"); old=base64.b64decode(os.environ["GLM53_CHAT_TEMPLATE_OLD_B64"]).decode(); new=base64.b64decode(os.environ["GLM53_CHAT_TEMPLATE_NEW_B64"]).decode(); text=raw.decode(); text.count(old)==1 or sys.exit("anvil patch: generation prompt anchor drift"); out=text.replace(old,new).encode(); hashlib.sha256(out).hexdigest()=="3133af873d620976b1561b76dc20ec2ad2599279ba61b873e872b1250f4f2ec5" or sys.exit("anvil patch: derived chat template hash mismatch"); dst.write_bytes(out)' "$SRC" "$DST"; echo 'anvil patch: WSL2 transport, logits gatherer, and thinking template ready'; exec sglang serve "$@" --chat-template "$DST"''',
  "glm53-adaptive-mtp",
]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--ormandj--GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO/snapshots/c3cbb9891b67c741bcbf6b176dd7af9265b069db"
quantization = "ModelOpt mixed W4A16 NVFP4 K32 routed experts plus FP8 weight-only attention/shared experts; FP8 E4M3 KV; BF16 recurrent state; native adaptive EAGLE MTP [3,5]"
context_tokens = 393216
port = 39118
startup_timeout_seconds = 3600
served_model_name = "glm53-flash-ormandj-sglang-sm120-tp2-393k-c1-adaptive-mtp"
ipc = "host"
shm_size = "64gb"
ulimits = ["memlock=-1", "nofile=1048576:1048576", "stack=67108864:67108864"]
named_volumes = ["glm53-ormandj-sglang-v60-rc14-adaptive-393k-c1-wsl2-cache:/root/.cache"]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "CUDA_VISIBLE_DEVICES=0,1",
  "PYTHONHASHSEED=0",
  "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False",
  "CUBLAS_WORKSPACE_CONFIG=:4096:2:16:8",
  "GLM53_CHAT_TEMPLATE_OLD_B64=eyUtIGlmIGFkZF9nZW5lcmF0aW9uX3Byb21wdCAtJX0KICAgIDx8YXNzaXN0YW50fD57ey0gJzx0aGluaz4nIC19fQp7JS0gZW5kaWYgLSV9",
  "GLM53_CHAT_TEMPLATE_NEW_B64=eyUtIGlmIGFkZF9nZW5lcmF0aW9uX3Byb21wdCAtJX0KICAgIHslLSBpZiBub3QgZW5hYmxlX3RoaW5raW5nIGlzIGRlZmluZWQgLSV9eyUtIHNldCBlbmFibGVfdGhpbmtpbmcgPSB0cnVlIC0lfXslLSBlbmRpZiAtJX0KICAgIDx8YXNzaXN0YW50fD57JS0gaWYgZW5hYmxlX3RoaW5raW5nIC0lfXt7LSAnPHRoaW5rPicgLX19eyUtIGVsc2UgLSV9e3stICc8dGhpbms+PC90aGluaz4nIC19fXslLSBlbmRpZiAtJX0KeyUtIGVuZGlmIC0lfQ==",
  "NCCL_CUMEM_ENABLE=0",
  "NCCL_DEBUG=INFO",
  "SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION=0",
  "TORCHINDUCTOR_CACHE_DIR=/root/.cache/torchinductor",
  "TILELANG_CACHE_DIR=/root/.cache/tilelang",
  "TRITON_CACHE_DIR=/root/.cache/triton",
]
flags = [
  "--served-model-name glm53-flash-ormandj-sglang-sm120-tp2-393k-c1-adaptive-mtp",
  "--tp-size 2",
  "--quantization modelopt_mixed",
  "--enable-multimodal",
  "--image-processor-backend torchvision",
  "--mm-process-config '{\"image\":{\"max_image_tokens\":3072}}'",
  "--warmups serving_coverage",
  "--moe-runner-backend flashinfer_cutlass",
  "--disable-shared-experts-fusion",
  "--disable-custom-all-reduce",
  "--attention-backend dsa",
  "--linear-attn-backend triton",
  "--dsa-prefill-backend flashinfer_sparse_mla",
  "--dsa-decode-backend flashinfer_sparse_mla",
  "--kv-cache-dtype fp8_e4m3",
  "--mamba-ssm-dtype bfloat16",
  "--context-length 393216",
  "--max-total-tokens 393216",
  "--mem-fraction-static 0.99",
  "--chunked-prefill-size 4096",
  "--max-prefill-tokens 4096",
  "--max-running-requests 1",
  "--max-mamba-cache-size 5",
  "--cuda-graph-max-bs-decode 1",
  "--speculative-algorithm EAGLE",
  "--speculative-num-steps 5",
  "--speculative-eagle-topk 1",
  "--speculative-num-draft-tokens 6",
  "--speculative-adaptive",
  "--speculative-adaptive-config /tmp/glm53-adaptive.json",
  "--reasoning-parser glm45",
  "--tool-call-parser glm47",
  "--enable-metrics",
  "--enable-cache-report",
  "--host 0.0.0.0",
  "--port 39118",
]
note = "Second fix-forward after the exact 499712-token/C4 profile left 49 MiB free per card and the 499712-token/C1 derivative left 2347 MiB free. This variant changes only the advertised context and shared token pool to 393216 while keeping one running request, five Mamba slots, and batch-size-one graphs. The pinned image, model revision, multimodal path, native adaptive [3,5] MTP, WSL2 transport fixes, hash-gated logits fallback, enable_thinking-aware template, quantization, KV dtype, parsers, and backends remain identical. Full functional, thinking-control, coding, image/OCR, 304491-token capacity, and 60-request endurance gates passed. The complete workload left 2101 MiB free per card with no OOM, CUDA error, crash, restart, corruption, or unexplained request loss. The operator explicitly waived the default 3072 MiB/card reserve for these exclusive model-only GPUs and separately authorized promotion on 2026-09-02."

[recipe.measured]
validated_prompt_tokens = 304491
post_workload_free_vram_mib_per_gpu = [2101, 2101]
default_reserve_mib_per_gpu = 3072
effective_reserve_mib_per_gpu = 0
reserve_waived = true
qualification = "full functional, thinking-control, coding, multimodal, capacity, and endurance gates passed; no OOM or crash"

[recipe.fit]
suited = ["qualification", "compatibility", "text", "tool-use", "structured-output", "multimodal", "long-context", "concurrency", "speculative-decoding"]
rationale = "Selected published long-context profile: it passed the complete qualification battery through 304491 actual prompt tokens and retained 2101 MiB/card after workload under the explicit model-only reserve waiver; managed, routed, and real-client promotion gates subsequently passed."

[recipe.sources]
official = [
  "https://huggingface.co/ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO/tree/c3cbb9891b67c741bcbf6b176dd7af9265b069db",
]
community = [
  "https://github.com/ormandj/sglang-glm53-flash-sm120/tree/a547c90c74f1363920287eb80adc88a16d1e7005",
]
local = [
  "configs/glm53-flash-ormandj-sglang-sm120-tp2-499k-c1-adaptive-mtp-recipe.toml",
  "configs/glm53-flash-ormandj-sglang-sm120-tp2-499k-adaptive-mtp-recipe.toml",
  "docs/findings/2026-09-02-glm53-sglang-sm120-qualification-evidence/run-plan.json",
  "docs/findings/2026-09-02-glm53-sglang-sm120-vram-policy-reclassification.json",
]

Adaptive MTP, 245,760 tokens — conservative fallback

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO"
status = "verified"
source = "2026-09-02 fully qualified 245760-token C1 conservative fallback, superseded as the selected challenger by the verified 393216-token profile"
family = "glm-5.3-flash-multimodal-moe"
total_params_b = 744
active_params_b = 40

[recipe.download]
repo = "ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO"
revision = "c3cbb9891b67c741bcbf6b176dd7af9265b069db"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 192
vram_note = "Aggregate PCIe VRAM without NVLink under WSL2; exclusive TP=2 only. This final fix-forward keeps one running request, five recurrent-state slots, and batch-size-one graphs, and reduces the shared context/token pool to 245760 so host-visible free VRAM remains above 3 GiB per card after deep text, multimodal, quality, and endurance workloads."

[recipe.serve]
engine = "sglang-glm53-sm120-v0.1.1-rc14"
allow_cuda_visible_devices_index = true
image = "ghcr.io/ormandj/sglang-glm53-flash-sm120@sha256:0c0637959c3931829f05154087bbefd2c50003fb9b2010200ce0ec82f4d71a53"
entrypoint = [
  "bash",
  "-lc",
  '''set -e; printf '%s\n' '{"1":{"candidate_steps":[3,5],"up_hysteresis":0.0,"down_hysteresis":-0.25,"ceiling_coeff":0},"4":{"candidate_steps":[3,5],"up_hysteresis":0.0,"down_hysteresis":-0.25,"ceiling_coeff":0}}' > /tmp/glm53-adaptive.json; LP=/opt/sglang-source/python/sglang/srt/layers/logits_processor.py; echo '05cfcbd9c1439af751a22fb03490a2e5a252ea4bf5ad5febdae1eb87dce1e9a7  /opt/sglang-source/python/sglang/srt/layers/logits_processor.py' | sha256sum -c -; test "$(grep -Fc 'enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,' "$LP")" = 1 || { echo 'anvil patch: logits gatherer anchor drift' >&2; exit 3; }; sed -i 's/enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,/enabled=False,/' "$LP"; grep -Fq 'enabled=False,' "$LP" || { echo 'anvil patch: logits gatherer postcondition failed' >&2; exit 4; }; SRC=/root/.cache/huggingface/hub/models--ormandj--GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO/snapshots/c3cbb9891b67c741bcbf6b176dd7af9265b069db/chat_template.jinja; DST=/root/.cache/glm53-ormandj-enable-thinking-chat-template.jinja; python3 -c 'import base64,hashlib,os,pathlib,sys; src=pathlib.Path(sys.argv[1]); dst=pathlib.Path(sys.argv[2]); raw=src.read_bytes(); hashlib.sha256(raw).hexdigest()=="34d5ee66b12fa6446cdae131c352b8f68cd85369e0e6fda115583805fada3891" or sys.exit("anvil patch: source chat template hash drift"); old=base64.b64decode(os.environ["GLM53_CHAT_TEMPLATE_OLD_B64"]).decode(); new=base64.b64decode(os.environ["GLM53_CHAT_TEMPLATE_NEW_B64"]).decode(); text=raw.decode(); text.count(old)==1 or sys.exit("anvil patch: generation prompt anchor drift"); out=text.replace(old,new).encode(); hashlib.sha256(out).hexdigest()=="3133af873d620976b1561b76dc20ec2ad2599279ba61b873e872b1250f4f2ec5" or sys.exit("anvil patch: derived chat template hash mismatch"); dst.write_bytes(out)' "$SRC" "$DST"; echo 'anvil patch: WSL2 transport, logits gatherer, and thinking template ready'; exec sglang serve "$@" --chat-template "$DST"''',
  "glm53-adaptive-mtp",
]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--ormandj--GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO/snapshots/c3cbb9891b67c741bcbf6b176dd7af9265b069db"
quantization = "ModelOpt mixed W4A16 NVFP4 K32 routed experts plus FP8 weight-only attention/shared experts; FP8 E4M3 KV; BF16 recurrent state; native adaptive EAGLE MTP [3,5]"
context_tokens = 245760
port = 39118
startup_timeout_seconds = 3600
served_model_name = "glm53-flash-ormandj-sglang-sm120-tp2-240k-c1-adaptive-mtp"
ipc = "host"
shm_size = "64gb"
ulimits = ["memlock=-1", "nofile=1048576:1048576", "stack=67108864:67108864"]
named_volumes = ["glm53-ormandj-sglang-v61-rc14-adaptive-240k-c1-wsl2-cache:/root/.cache"]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "CUDA_VISIBLE_DEVICES=0,1",
  "PYTHONHASHSEED=0",
  "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False",
  "CUBLAS_WORKSPACE_CONFIG=:4096:2:16:8",
  "GLM53_CHAT_TEMPLATE_OLD_B64=eyUtIGlmIGFkZF9nZW5lcmF0aW9uX3Byb21wdCAtJX0KICAgIDx8YXNzaXN0YW50fD57ey0gJzx0aGluaz4nIC19fQp7JS0gZW5kaWYgLSV9",
  "GLM53_CHAT_TEMPLATE_NEW_B64=eyUtIGlmIGFkZF9nZW5lcmF0aW9uX3Byb21wdCAtJX0KICAgIHslLSBpZiBub3QgZW5hYmxlX3RoaW5raW5nIGlzIGRlZmluZWQgLSV9eyUtIHNldCBlbmFibGVfdGhpbmtpbmcgPSB0cnVlIC0lfXslLSBlbmRpZiAtJX0KICAgIDx8YXNzaXN0YW50fD57JS0gaWYgZW5hYmxlX3RoaW5raW5nIC0lfXt7LSAnPHRoaW5rPicgLX19eyUtIGVsc2UgLSV9e3stICc8dGhpbms+PC90aGluaz4nIC19fXslLSBlbmRpZiAtJX0KeyUtIGVuZGlmIC0lfQ==",
  "NCCL_CUMEM_ENABLE=0",
  "NCCL_DEBUG=INFO",
  "SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION=0",
  "TORCHINDUCTOR_CACHE_DIR=/root/.cache/torchinductor",
  "TILELANG_CACHE_DIR=/root/.cache/tilelang",
  "TRITON_CACHE_DIR=/root/.cache/triton",
]
flags = [
  "--served-model-name glm53-flash-ormandj-sglang-sm120-tp2-240k-c1-adaptive-mtp",
  "--tp-size 2",
  "--quantization modelopt_mixed",
  "--enable-multimodal",
  "--image-processor-backend torchvision",
  "--mm-process-config '{\"image\":{\"max_image_tokens\":3072}}'",
  "--warmups serving_coverage",
  "--moe-runner-backend flashinfer_cutlass",
  "--disable-shared-experts-fusion",
  "--disable-custom-all-reduce",
  "--attention-backend dsa",
  "--linear-attn-backend triton",
  "--dsa-prefill-backend flashinfer_sparse_mla",
  "--dsa-decode-backend flashinfer_sparse_mla",
  "--kv-cache-dtype fp8_e4m3",
  "--mamba-ssm-dtype bfloat16",
  "--context-length 245760",
  "--max-total-tokens 245760",
  "--mem-fraction-static 0.99",
  "--chunked-prefill-size 4096",
  "--max-prefill-tokens 4096",
  "--max-running-requests 1",
  "--max-mamba-cache-size 5",
  "--cuda-graph-max-bs-decode 1",
  "--speculative-algorithm EAGLE",
  "--speculative-num-steps 5",
  "--speculative-eagle-topk 1",
  "--speculative-num-draft-tokens 6",
  "--speculative-adaptive",
  "--speculative-adaptive-config /tmp/glm53-adaptive.json",
  "--reasoning-parser glm45",
  "--tool-call-parser glm47",
  "--enable-metrics",
  "--enable-cache-report",
  "--host 0.0.0.0",
  "--port 39118",
]
note = "Conservative fallback after the exact 499712-token/C4 profile left 49 MiB free per card, the 499712-token/C1 derivative left 2347 MiB at startup, and the 393216-token/C1 derivative fell from 3351 to 2101 MiB after the full workload battery. This variant changes only the advertised context and shared token pool to 245760 while keeping one running request, five Mamba slots, and batch-size-one graphs. It passed the complete qualification battery and retained 3487 MiB/card after workload. It remains verified, but the 393216-token profile supersedes it as the selected challenger under the operator's explicit model-only reserve waiver. Router and promotion remain separately gated."

[recipe.fit]
suited = ["qualification", "compatibility", "text", "tool-use", "structured-output", "multimodal", "long-context", "concurrency", "speculative-decoding"]
rationale = "Verified conservative fallback that preserves at least 3 GiB/card after workload; the 393216-token profile is the selected challenger under an explicit model-only reserve waiver."

[recipe.sources]
official = [
  "https://huggingface.co/ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO/tree/c3cbb9891b67c741bcbf6b176dd7af9265b069db",
]
community = [
  "https://github.com/ormandj/sglang-glm53-flash-sm120/tree/a547c90c74f1363920287eb80adc88a16d1e7005",
]
local = [
  "configs/glm53-flash-ormandj-sglang-sm120-tp2-393k-c1-adaptive-mtp-recipe.toml",
  "configs/glm53-flash-ormandj-sglang-sm120-tp2-499k-c1-adaptive-mtp-recipe.toml",
  "configs/glm53-flash-ormandj-sglang-sm120-tp2-499k-adaptive-mtp-recipe.toml",
  "docs/findings/2026-09-02-glm53-sglang-sm120-qualification-evidence/run-plan.json",
  "docs/findings/2026-09-02-glm53-sglang-sm120-vram-policy-reclassification.json",
]

Matched no-speculation 131K control

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO"
status = "qualification-control"
source = "2026-09-02 WSL2 NCCL-cuMem-off no-speculation control translated from ormandj/sglang-glm53-flash-sm120 v0.1.1-rc.14"
family = "glm-5.3-flash-multimodal-moe"
total_params_b = 744
active_params_b = 40

[recipe.download]
repo = "ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO"
revision = "c3cbb9891b67c741bcbf6b176dd7af9265b069db"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 192
vram_note = "Aggregate PCIe VRAM without NVLink under WSL2; exclusive TP=2 only. The upstream stable run used the same GPU product and PCIe topology under native Linux, so transport and reserve are requalified locally."

[recipe.serve]
engine = "sglang-glm53-sm120-v0.1.1-rc14"
allow_cuda_visible_devices_index = true
image = "ghcr.io/ormandj/sglang-glm53-flash-sm120@sha256:0c0637959c3931829f05154087bbefd2c50003fb9b2010200ce0ec82f4d71a53"
entrypoint = [
  "bash",
  "-lc",
  '''set -e; LP=/opt/sglang-source/python/sglang/srt/layers/logits_processor.py; echo '05cfcbd9c1439af751a22fb03490a2e5a252ea4bf5ad5febdae1eb87dce1e9a7  /opt/sglang-source/python/sglang/srt/layers/logits_processor.py' | sha256sum -c -; test "$(grep -Fc 'enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,' "$LP")" = 1 || { echo 'anvil patch: logits gatherer anchor drift' >&2; exit 3; }; sed -i 's/enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,/enabled=False,/' "$LP"; grep -Fq 'enabled=False,' "$LP" || { echo 'anvil patch: logits gatherer postcondition failed' >&2; exit 4; }; echo 'anvil patch: symmetric-memory logits gatherer disabled; NCCL fallback selected'; SRC=/root/.cache/huggingface/hub/models--ormandj--GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO/snapshots/c3cbb9891b67c741bcbf6b176dd7af9265b069db/chat_template.jinja; DST=/root/.cache/glm53-ormandj-enable-thinking-chat-template.jinja; python3 -c 'import base64,hashlib,os,pathlib,sys; src=pathlib.Path(sys.argv[1]); dst=pathlib.Path(sys.argv[2]); raw=src.read_bytes(); expected="34d5ee66b12fa6446cdae131c352b8f68cd85369e0e6fda115583805fada3891"; actual=hashlib.sha256(raw).hexdigest(); actual==expected or sys.exit("anvil patch: source chat template hash drift"); old=base64.b64decode(os.environ["GLM53_CHAT_TEMPLATE_OLD_B64"]).decode(); new=base64.b64decode(os.environ["GLM53_CHAT_TEMPLATE_NEW_B64"]).decode(); text=raw.decode(); text.count(old)==1 or sys.exit("anvil patch: generation prompt anchor drift"); out=text.replace(old,new).encode(); hashlib.sha256(out).hexdigest()=="3133af873d620976b1561b76dc20ec2ad2599279ba61b873e872b1250f4f2ec5" or sys.exit("anvil patch: derived chat template hash mismatch"); dst.write_bytes(out)' "$SRC" "$DST"; echo 'anvil patch: enable_thinking-aware chat template derived from immutable snapshot'; exec sglang serve "$@" --chat-template "$DST"''',
  "glm53-sglang-sm120",
]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--ormandj--GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO/snapshots/c3cbb9891b67c741bcbf6b176dd7af9265b069db"
quantization = "ModelOpt mixed W4A16 NVFP4 K32 routed experts plus FP8 weight-only attention/shared experts; FP8 E4M3 KV; BF16 recurrent state"
context_tokens = 131072
port = 39118
startup_timeout_seconds = 3600
served_model_name = "glm53-flash-ormandj-sglang-sm120-tp2-131k-nospec"
ipc = "host"
shm_size = "64gb"
ulimits = ["memlock=-1", "nofile=1048576:1048576", "stack=67108864:67108864"]
named_volumes = ["glm53-ormandj-sglang-v57-rc14-nospec-131k-wsl2-cache:/root/.cache"]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "CUDA_VISIBLE_DEVICES=0,1",
  "PYTHONHASHSEED=0",
  "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False",
  "CUBLAS_WORKSPACE_CONFIG=:4096:2:16:8",
  "GLM53_CHAT_TEMPLATE_OLD_B64=eyUtIGlmIGFkZF9nZW5lcmF0aW9uX3Byb21wdCAtJX0KICAgIDx8YXNzaXN0YW50fD57ey0gJzx0aGluaz4nIC19fQp7JS0gZW5kaWYgLSV9",
  "GLM53_CHAT_TEMPLATE_NEW_B64=eyUtIGlmIGFkZF9nZW5lcmF0aW9uX3Byb21wdCAtJX0KICAgIHslLSBpZiBub3QgZW5hYmxlX3RoaW5raW5nIGlzIGRlZmluZWQgLSV9eyUtIHNldCBlbmFibGVfdGhpbmtpbmcgPSB0cnVlIC0lfXslLSBlbmRpZiAtJX0KICAgIDx8YXNzaXN0YW50fD57JS0gaWYgZW5hYmxlX3RoaW5raW5nIC0lfXt7LSAnPHRoaW5rPicgLX19eyUtIGVsc2UgLSV9e3stICc8dGhpbms+PC90aGluaz4nIC19fXslLSBlbmRpZiAtJX0KeyUtIGVuZGlmIC0lfQ==",
  "NCCL_CUMEM_ENABLE=0",
  "NCCL_DEBUG=INFO",
  "SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION=0",
  "TORCHINDUCTOR_CACHE_DIR=/root/.cache/torchinductor",
  "TILELANG_CACHE_DIR=/root/.cache/tilelang",
  "TRITON_CACHE_DIR=/root/.cache/triton",
]
flags = [
  "--served-model-name glm53-flash-ormandj-sglang-sm120-tp2-131k-nospec",
  "--tp-size 2",
  "--quantization modelopt_mixed",
  "--enable-multimodal",
  "--image-processor-backend torchvision",
  "--mm-process-config '{\"image\":{\"max_image_tokens\":3072}}'",
  "--warmups serving_coverage",
  "--moe-runner-backend flashinfer_cutlass",
  "--disable-shared-experts-fusion",
  "--disable-custom-all-reduce",
  "--attention-backend dsa",
  "--linear-attn-backend triton",
  "--dsa-prefill-backend flashinfer_sparse_mla",
  "--dsa-decode-backend flashinfer_sparse_mla",
  "--kv-cache-dtype fp8_e4m3",
  "--mamba-ssm-dtype bfloat16",
  "--context-length 131072",
  "--max-total-tokens 131072",
  "--mem-fraction-static 0.99",
  "--chunked-prefill-size 4096",
  "--max-prefill-tokens 4096",
  "--max-running-requests 1",
  "--max-mamba-cache-size 5",
  "--cuda-graph-max-bs-decode 1",
  "--reasoning-parser glm45",
  "--tool-call-parser glm47",
  "--enable-metrics",
  "--enable-cache-report",
  "--host 0.0.0.0",
  "--port 39118",
]
note = "First-boot and matched no-speculation control for the v0.1.1-rc.14 fix-forward image. It deliberately disables HiCache, all CPU/KV offload, and the upstream opt-in PCIe IPC allreduce after the exact enabled profile failed both TP ranks during their first distributed CUDA allocation under WSL2 while a minimal two-GPU CUDA probe passed. The otherwise identical retry failed at the same boundary. Isolated two-rank probes then proved two independent WSL2 controls: NCCL 2.30.7 failed in ncclCuMemMapAndSetAccess until NCCL_CUMEM_ENABLE=0, and the upstream expandable CUDA allocator still made the first rank-local tensor fail until expandable_segments was set false. With those fixed, the model loaded and tuned but Torch symmetric-memory rendezvous raised SIGFPE in the logits all-gather during decode-graph capture. The entrypoint verifies the exact source hash and one expected anchor, disables only that optimization, verifies the postcondition, and retains SGLang's normal NCCL all-gather fallback. The checkpoint chat template always opened a think block and ignored enable_thinking, which leaked hidden reasoning into content and broke strict JSON and Responses checks. The same entrypoint now verifies the immutable template hash and exact generation-prompt anchor, derives a hashed runtime-cache copy that defaults thinking on but emits an empty think block when enable_thinking is false, and supplies it with --chat-template without modifying the checkpoint. It caps the shared token pool and context at 131072, limits concurrency to one, allocates five recurrent-state slots, and captures only batch size one. Router and promotion state remain out of scope."

[recipe.fit]
suited = ["qualification", "compatibility", "text", "tool-use", "structured-output", "multimodal-control"]
rationale = "Establishes local WSL2 correctness and a matched target-only performance control before adaptive native MTP or maximum-context testing."

[recipe.sources]
official = [
  "https://huggingface.co/ormandj/GLM-5.3-Flash-W4A16-NVFP4-K32-Experts-FP8-WO/tree/c3cbb9891b67c741bcbf6b176dd7af9265b069db",
]
community = [
  "https://github.com/ormandj/sglang-glm53-flash-sm120/tree/a547c90c74f1363920287eb80adc88a16d1e7005",
]
local = [
  "docs/findings/2026-09-02-glm53-sglang-sm120-qualification-evidence/run-plan.json",
]

Additional retained recipes: 131K adaptive, 499K/C1 unverified, and 499K/C4 rejected.

Qwen3.8 Flash Next MTP3 and control

This otherwise-matched pair pins RadixArk revision 7b719225242aacd3dbd3f9407468c2ee9a9d2594, SGLang image digest 59f06adce6f91401adf443bd168d45fdb2044d77671fd591c7c57a29d851cbae, TP=2, BF16 KV/recurrent state, 262,144-token context, concurrency one, 0.80 static memory, CPU-offload exclusions, and the same guarded SM120 QSA fast-path entrypoint. The MTP arm adds NEXTN steps/top-k/draft tokens 3/1/4.

These files declare vllm-hfcache for the model snapshot and no additional named runtime volume. The entrypoint source hashes are part of the configuration: a newer SGLang file that fails either hash check is not an equivalent run.

Performance and promotion record · Vision qualification · Model dossier

MTP3 — full tracked recipe

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "RadixArk/Qwen3.8-Flash-Next-NVFP4"
status = "qualification-candidate"
source = "2026-08-26 matched NEXTN MTP3 arm for the upstream SGLang PR 36556 SM120 QSA fast path"
family = "qwen3.8-flash-next-multimodal-moe"
total_params_b = 180

[recipe.download]
repo = "RadixArk/Qwen3.8-Flash-Next-NVFP4"
revision = "7b719225242aacd3dbd3f9407468c2ee9a9d2594"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 192
vram_note = "Aggregate PCIe VRAM without NVLink under WSL2; exclusive TP=2 compatibility experiment. Publisher validation used B200/B300/GB300-class native Linux hardware, not this host."

[recipe.serve]
engine = "sglang-qwen38-flash-next-day-zero-pr36556"
allow_cuda_visible_devices_index = true
image = "lmsysorg/sglang@sha256:59f06adce6f91401adf443bd168d45fdb2044d77671fd591c7c57a29d851cbae"
entrypoint = [
  "bash",
  "-lc",
  '''set -e; grep -q 'enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,' /sgl-workspace/sglang/python/sglang/srt/layers/logits_processor.py || { echo 'anvil patch: logits gatherer anchor drift' >&2; exit 3; }; sed -i 's/enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,/enabled=False,/' /sgl-workspace/sglang/python/sglang/srt/layers/logits_processor.py; grep -q 'enabled=False,' /sgl-workspace/sglang/python/sglang/srt/layers/logits_processor.py || { echo 'anvil patch: logits gatherer postcondition failed' >&2; exit 4; }; echo 'c959835d05d0f395ad7eae4330cf264af9f6f7c1bff3d45a39bb953d2536f5f2  /sgl-workspace/sglang/python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py' | sha256sum -c -; sed -i 's/from sglang.srt.utils import is_sm100_supported/from sglang.srt.utils import is_sm100_supported, is_sm120_supported/' /sgl-workspace/sglang/python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py; sed -i 's/if not is_sm100_supported():/if not (is_sm100_supported() or is_sm120_supported()):/' /sgl-workspace/sglang/python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py; echo 'a6b003ed21b3be8ba763e8627aee39baee3d84184f5bf0fc650a1a6b853119d3  /sgl-workspace/sglang/python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py' | sha256sum -c -; python3 -c "from flashinfer.decode import trtllm_batch_decode_with_kv_cache; print('anvil patch: exact SGLang PR 36556 SM120 TRTLLM QSA fast path ready')"; exec python3 -m sglang.launch_server "$@"''',
  "qwen38-flash-next-qsa-fast-mtp3",
]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--RadixArk--Qwen3.8-Flash-Next-NVFP4/snapshots/7b719225242aacd3dbd3f9407468c2ee9a9d2594"
quantization = "NVIDIA ModelOpt NVFP4 W4A4 routed experts; BF16 attention, GDN, QSA, shared experts, vision, and MTP; FP8 PLE tables dequantized to BF16 at load"
context_tokens = 262144
port = 39081
startup_timeout_seconds = 2400
served_model_name = "qwen38-flash-next-radixark-nvfp4-sglang-qsa-fast-tp2-262k-mtp3"
ipc = "host"
shm_size = "32gb"
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTHONHASHSEED=0",
  "SGLANG_USE_CUDA_IPC_TRANSPORT=0",
  "NCCL_IB_DISABLE=1",
  "NCCL_P2P_DISABLE=1",
  "NCCL_CUMEM_ENABLE=0",
  "NCCL_CUMEM_HOST_ENABLE=0",
  "NCCL_DEBUG=INFO",
]
flags = [
  "--served-model-name qwen38-flash-next-radixark-nvfp4-sglang-qsa-fast-tp2-262k-mtp3",
  "--tp 2",
  "--quantization modelopt_fp4",
  "--fp4-gemm-backend flashinfer_cutlass",
  "--attention-backend triton",
  "--linear-attn-prefill-backend triton",
  "--linear-attn-decode-backend flashinfer",
  "--mamba-ssm-dtype bfloat16",
  "--page-size 64",
  "--mamba-scheduler-strategy extra_buffer",
  "--mamba-track-interval 64",
  "--chunked-prefill-size 4096",
  "--max-running-requests 1",
  "--context-length 262144",
  "--mem-fraction-static 0.80",
  "--no-ple-offload-embedding",
  "--disable-custom-all-reduce",
  "--cuda-graph-max-bs-decode 1",
  "--speculative-algorithm NEXTN",
  "--speculative-num-steps 3",
  "--speculative-eagle-topk 1",
  "--speculative-num-draft-tokens 4",
  "--reasoning-parser qwen3",
  "--tool-call-parser qwen3_coder",
  "--host 0.0.0.0",
  "--port 39081",
]
note = "Matched MTP3 arm for the PR-36556 QSA fast-path recipe. Model, revision, image, primary-path patch and hashes, TP2, 262144 context, BF16 KV, memory fraction, backends, batching, graph cap, transport, parsers, and offload policy match the no-speculation arm. The only intended performance variable is SGLang NEXTN with three steps, top-k one, and four draft tokens, matching SGLang's official low-latency cookbook. Credit it only if matched end-to-end work improves and every deterministic and repeated quality gate remains passing. FP8 KV and CPU PLE offload remain excluded. This recipe is not promotion authority."

[recipe.fit]
suited = ["qualification", "compatibility", "long-context", "coding-agent", "tool-use", "low-latency", "speculative-decoding"]
rationale = "Measures the official NEXTN/MTP3 low-latency preset only after the exact PR-36556 target-only control is qualified."

[recipe.sources]
official = [
  "https://github.com/sgl-project/sglang/pull/36556/commits/dac5523d1e5d2f4297fec40ef02fc76fb0f662d1",
  "https://github.com/sgl-project/sglang/issues/36531",
  "https://github.com/sgl-project/sglang/issues/36532",
  "https://github.com/sgl-project/sglang/issues/36545",
  "https://github.com/sgl-project/sglang/pull/36496",
  "https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4/tree/7b719225242aacd3dbd3f9407468c2ee9a9d2594",
]
local = [
  "configs/qwen38-flash-next-radixark-nvfp4-sglang-sm120-qsa-fast-tp2-262k-nospec-recipe.toml",
  "docs/findings/2026-08-26-qwen38-flash-next-qsa-fast-mtp3-promotion.md",
  "docs/findings/2026-08-26-qwen38-flash-next-qsa-fast-mtp3-evidence/summary.json",
]

No speculation — full tracked control

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "RadixArk/Qwen3.8-Flash-Next-NVFP4"
status = "qualification-candidate"
source = "2026-08-26 upstream SGLang PR 36556 SM120 QSA primary fast-path translation of the promoted TP2 control"
family = "qwen3.8-flash-next-multimodal-moe"
total_params_b = 180

[recipe.download]
repo = "RadixArk/Qwen3.8-Flash-Next-NVFP4"
revision = "7b719225242aacd3dbd3f9407468c2ee9a9d2594"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "2x NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 192
vram_note = "Aggregate PCIe VRAM without NVLink under WSL2; exclusive TP=2 compatibility experiment. Publisher validation used B200/B300/GB300-class native Linux hardware, not this host."

[recipe.serve]
engine = "sglang-qwen38-flash-next-day-zero-pr36556"
allow_cuda_visible_devices_index = true
image = "lmsysorg/sglang@sha256:59f06adce6f91401adf443bd168d45fdb2044d77671fd591c7c57a29d851cbae"
entrypoint = [
  "bash",
  "-lc",
  '''set -e; grep -q 'enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,' /sgl-workspace/sglang/python/sglang/srt/layers/logits_processor.py || { echo 'anvil patch: logits gatherer anchor drift' >&2; exit 3; }; sed -i 's/enabled=self.do_tensor_parallel_all_gather and not self.use_attn_tp_group,/enabled=False,/' /sgl-workspace/sglang/python/sglang/srt/layers/logits_processor.py; grep -q 'enabled=False,' /sgl-workspace/sglang/python/sglang/srt/layers/logits_processor.py || { echo 'anvil patch: logits gatherer postcondition failed' >&2; exit 4; }; echo 'c959835d05d0f395ad7eae4330cf264af9f6f7c1bff3d45a39bb953d2536f5f2  /sgl-workspace/sglang/python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py' | sha256sum -c -; sed -i 's/from sglang.srt.utils import is_sm100_supported/from sglang.srt.utils import is_sm100_supported, is_sm120_supported/' /sgl-workspace/sglang/python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py; sed -i 's/if not is_sm100_supported():/if not (is_sm100_supported() or is_sm120_supported()):/' /sgl-workspace/sglang/python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py; echo 'a6b003ed21b3be8ba763e8627aee39baee3d84184f5bf0fc650a1a6b853119d3  /sgl-workspace/sglang/python/sglang/srt/layers/attention/qwen_sparse_attn_backend.py' | sha256sum -c -; python3 -c "from flashinfer.decode import trtllm_batch_decode_with_kv_cache; print('anvil patch: exact SGLang PR 36556 SM120 TRTLLM QSA fast path ready')"; exec python3 -m sglang.launch_server "$@"''',
  "qwen38-flash-next-qsa-fast",
]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--RadixArk--Qwen3.8-Flash-Next-NVFP4/snapshots/7b719225242aacd3dbd3f9407468c2ee9a9d2594"
quantization = "NVIDIA ModelOpt NVFP4 W4A4 routed experts; BF16 attention, GDN, QSA, shared experts, vision, and MTP; FP8 PLE tables dequantized to BF16 at load"
context_tokens = 262144
port = 39080
startup_timeout_seconds = 2400
served_model_name = "qwen38-flash-next-radixark-nvfp4-sglang-qsa-fast-tp2-262k-nospec"
ipc = "host"
shm_size = "32gb"
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTHONHASHSEED=0",
  "SGLANG_USE_CUDA_IPC_TRANSPORT=0",
  "NCCL_IB_DISABLE=1",
  "NCCL_P2P_DISABLE=1",
  "NCCL_CUMEM_ENABLE=0",
  "NCCL_CUMEM_HOST_ENABLE=0",
  "NCCL_DEBUG=INFO",
]
flags = [
  "--served-model-name qwen38-flash-next-radixark-nvfp4-sglang-qsa-fast-tp2-262k-nospec",
  "--tp 2",
  "--quantization modelopt_fp4",
  "--fp4-gemm-backend flashinfer_cutlass",
  "--attention-backend triton",
  "--linear-attn-prefill-backend triton",
  "--linear-attn-decode-backend flashinfer",
  "--mamba-ssm-dtype bfloat16",
  "--page-size 64",
  "--mamba-scheduler-strategy extra_buffer",
  "--mamba-track-interval 64",
  "--chunked-prefill-size 4096",
  "--max-running-requests 1",
  "--context-length 262144",
  "--mem-fraction-static 0.80",
  "--no-ple-offload-embedding",
  "--disable-custom-all-reduce",
  "--cuda-graph-max-bs-decode 1",
  "--reasoning-parser qwen3",
  "--tool-call-parser qwen3_coder",
  "--host 0.0.0.0",
  "--port 39080",
]
note = "Matched target-only speed candidate for the promoted RadixArk/SGLang lane. It replaces the portable QSA sparse-decode reference with the primary SM120 TRTLLM decode gate from upstream SGLang PR 36556 at head dac5523d1e5d2f4297fec40ef02fc76fb0f662d1. Startup verifies the exact source hash before and after the pinned hunk and fails unless the required FlashInfer decode symbol imports. It selects Triton attention and prefill plus FlashInfer GDN decode, BF16 recurrent state, and bounded batch-one CUDA graph capture. TP2, 262144 context, concurrency one, BF16 KV, 0.80 static memory, no PLE CPU offload, and the proven NCCL controls stay matched. FP8 KV is excluded because SGLang issue 36545 records an SM120 QSA dtype failure. MTP remains off for the causal control. External exact-hardware reports of 94.5 decode tok/s and 5/5 retrieval near 229.5K are recipe priors, not local evidence. This recipe is not promotion authority."

[recipe.fit]
suited = ["qualification", "compatibility", "long-context", "coding-agent", "tool-use", "low-latency"]
rationale = "Tests the upstream SM120 QSA kernel fix against the exact slow promoted control before adding speculation."

[recipe.sources]
official = [
  "https://github.com/sgl-project/sglang/pull/36556/commits/dac5523d1e5d2f4297fec40ef02fc76fb0f662d1",
  "https://github.com/sgl-project/sglang/issues/36531",
  "https://github.com/sgl-project/sglang/issues/36532",
  "https://github.com/sgl-project/sglang/issues/36545",
  "https://github.com/sgl-project/sglang/pull/36496",
  "https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4/tree/7b719225242aacd3dbd3f9407468c2ee9a9d2594",
]
local = [
  "docs/findings/2026-08-26-qwen38-flash-next-qsa-fast-mtp3-promotion.md",
  "docs/findings/2026-08-26-qwen38-flash-next-qsa-fast-mtp3-evidence/summary.json",
]

Qwen3.8 27B official FP8 SGLang

The reusable multimodal profile pins official model revision 017b9c7af6b5689d5dd426a76e0bc077eb5ca20a, SGLang digest 506525a5907ea22c9d445afb7c03603959b912de034d86915cf17da814f1a124, TP=1, 393,216-token context, concurrency one, FP8 E4M3 KV, FlashInfer attention, 2K chunks, disabled prefix cache, thinking disabled by default, and the qwen38-fp8-sglang-cache runtime volume. The multimodal profile uses CPU feature transport, five GDN states, and EAGLE 3/1/4; the no-spec campaign control is language-only with one GDN state.

Because media transport, modality, and GDN capacity also differ, these two files are not by themselves a one-variable speculation A/B. Use the dated finding for the exact matched measurement arms rather than deriving a speedup from these reusable endpoint definitions.

MTP and multimodal qualification · Video expansion · Model dossier

MTP3 multimodal with CPU feature transport — full tracked recipe

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "Qwen/Qwen3.8-27B-FP8"
status = "qualification-candidate"
source = "2026-08-15 SGLang TP1 393K MTP=3 multimodal CPU-transport diagnostic arm"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Qwen/Qwen3.8-27B-FP8"
revision = "017b9c7af6b5689d5dd426a76e0bc077eb5ca20a"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "One GPU in split mode; multimodal feature transfer uses host memory rather than CUDA IPC."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:506525a5907ea22c9d445afb7c03603959b912de034d86915cf17da814f1a124"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Qwen--Qwen3.8-27B-FP8/snapshots/017b9c7af6b5689d5dd426a76e0bc077eb5ca20a"
quantization = "official block-scaled FP8 weights with explicit FP8 E4M3 KV cache"
context_tokens = 393216
port = 39215
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-official-fp8-sglang-tp1-393k-mtp3-mm-cpu"
ipc = "host"
shm_size = "32gb"
named_volumes = [
  "qwen38-fp8-sglang-cache:/root/.cache/sglang",
]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTHONHASHSEED=0",
  "SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1",
  "SGLANG_USE_CUDA_IPC_TRANSPORT=0",
  "SGLANG_VIT_ENABLE_CUDA_GRAPH=0",
  "SGLANG_MM_BUFFER_SIZE_MB=0",
]
flags = [
  "--served-model-name qwen38-27b-official-fp8-sglang-tp1-393k-mtp3-mm-cpu",
  "--tp-size 1",
  "--mm-feature-transport cpu",
  "--context-length 393216",
  "--max-running-requests 1",
  "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3",
  "--attention-backend flashinfer",
  "--chunked-prefill-size 2048",
  "--disable-radix-cache",
  "--max-mamba-cache-size 5",
  "--speculative-algorithm EAGLE",
  "--speculative-num-steps 3",
  "--speculative-eagle-topk 1",
  "--speculative-num-draft-tokens 4",
  "--reasoning-parser qwen3",
  "--tool-call-parser qwen3_coder",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'",
  "--host 0.0.0.0",
  "--port 39215",
]
note = "Exact portable source for the human-promoted 2026-08-15 single-service profile. The prior image request failed while SGLang reconstructed its automatically selected CUDA-IPC feature pool with CUDA invalid-resource-handle errors. This recipe removes --language-only, explicitly selects the supported CPU feature transport, disables CUDA-IPC transport defensively, and disables the initial vision CUDA-graph and multimodal buffer optimizations. Checkpoint, image, TP, context, concurrency, KV dtype, attention, prefill, Mamba cache, MTP, parsers, and thinking mode remain fixed. A later deterministic corpus qualified one video per request while retaining the two-image and concurrency-one ceiling. This public recipe records the configuration but does not itself enact a promotion or route change."

[recipe.fit]
suited = ["qualification", "long-context", "multimodal", "vision", "ocr", "video", "speculative-decoding"]
rationale = "CPU feature transport removes the observed WSL2/Docker CUDA-IPC failure while retaining the matched MTP configuration; the qualified envelope includes deterministic image, video, and mixed-media requests."

[recipe.sources]
official = [
  "https://github.com/sgl-project/sglang/blob/dd458f3212dd4ddf0e1a7907bbf539b660e70d21/docs/cookbook/autoregressive/Qwen/Qwen3.8-27B.mdx",
  "https://github.com/sgl-project/sglang/blob/c4271c3fe1262fc2adbd162c33b25de5255251c5/python/sglang/srt/server_args.py",
  "https://github.com/sgl-project/sglang/pull/33899",
  "https://huggingface.co/Qwen/Qwen3.8-27B-FP8/tree/017b9c7af6b5689d5dd426a76e0bc077eb5ca20a",
]
local = [
  "configs/qwen38-27b-official-fp8-sglang-tp1-393k-mtp3-recipe.toml",
  "docs/findings/2026-08-15-qwen38-27b-sglang-nvfp4-qualification.md",
]

No-speculation text control — full tracked recipe

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "Qwen/Qwen3.8-27B-FP8"
status = "qualification-candidate"
source = "2026-08-15 SGLang RTX PRO 6000 recipe translated to the existing TP1 393K no-speculation control shape"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "Qwen/Qwen3.8-27B-FP8"
revision = "017b9c7af6b5689d5dd426a76e0bc077eb5ca20a"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition"
arch = "sm_120"
vram_total_gb = 96
vram_note = "One GPU in split mode; the other equal GPU remains independently assigned."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang@sha256:506525a5907ea22c9d445afb7c03603959b912de034d86915cf17da814f1a124"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--Qwen--Qwen3.8-27B-FP8/snapshots/017b9c7af6b5689d5dd426a76e0bc077eb5ca20a"
quantization = "official block-scaled FP8 weights with explicit FP8 E4M3 KV cache"
context_tokens = 393216
port = 39211
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-official-fp8-sglang-tp1-393k-control"
ipc = "host"
shm_size = "32gb"
named_volumes = [
  "qwen38-fp8-sglang-cache:/root/.cache/sglang",
]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTHONHASHSEED=0",
  "SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1",
]
flags = [
  "--served-model-name qwen38-27b-official-fp8-sglang-tp1-393k-control",
  "--tp-size 1",
  "--language-only",
  "--context-length 393216",
  "--max-running-requests 1",
  "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3",
  "--attention-backend flashinfer",
  "--chunked-prefill-size 2048",
  "--disable-radix-cache",
  "--max-mamba-cache-size 1",
  "--reasoning-parser qwen3",
  "--tool-call-parser qwen3_coder",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'",
  "--host 0.0.0.0",
  "--port 39211",
]
note = "Matched SGLang no-speculation control for the TP1 393K campaign. The upstream RTX PRO 6000 cell supplies mem-fraction 0.85, FlashInfer attention, 2K chunked prefill, and Qwen parsers. This translation adds the campaign's explicit 393K/c1/FP8-KV shape and --language-only to match the existing vLLM FP8 text control. SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 is SGLang's fail-closed opt-in for exceeding the checkpoint's advertised 262K native limit, equivalent in intent to the nested config override in the existing vLLM 393K control. The default chat-template kwargs keep thinking disabled across both Chat Completions and SGLang's Responses adapter; request-level controls remain testable overrides. Prefix caching is disabled and one GDN state slot is allocated because the qualification workload has no shared-prefix reuse. A model-specific named volume retains generated SGLang/FlashInfer caches across card-swap restarts without sharing writes between models. The upstream cell includes --trust-remote-code, but the pinned official snapshot has no tracked Python or auto_map hook, and this recipe deliberately omits executable remote-code permission. The immutable image label reports SGLang c4271c3fe1262fc2adbd162c33b25de5255251c5 while an internal build-version string reports 561c8f3; retain that provenance discrepancy as a caveat. Router and promotion state remain out of scope."

[recipe.fit]
suited = ["qualification", "long-context", "text", "coding", "agentic"]
rationale = "Measures the SGLang runtime against the existing official-FP8 vLLM control on one SM120 card without changing a routed alias."

[recipe.sources]
official = [
  "https://github.com/sgl-project/sglang/blob/dd458f3212dd4ddf0e1a7907bbf539b660e70d21/docs/cookbook/autoregressive/Qwen/Qwen3.8-27B.mdx",
  "https://github.com/sgl-project/sglang/blob/dd458f3212dd4ddf0e1a7907bbf539b660e70d21/docs/src/snippets/configs/Qwen/qwen3.8-27b.jsx",
  "https://huggingface.co/Qwen/Qwen3.8-27B-FP8/tree/017b9c7af6b5689d5dd426a76e0bc077eb5ca20a",
]
local = [
  "configs/qwen38-27b-fp8-tp1-393k-control-recipe.toml",
  "docs/findings/2026-08-15-qwen38-27b-external-recipe-refresh.md",
  "docs/findings/2026-08-15-qwen38-27b-sglang-nvfp4-qualification.md",
]

Qwen3.8 27B RadixArk RTX 5090 128K multimodal

This direct-only profile pins RadixArk revision 554ebba9b5f1b79dc11246341960360e6ef05ef4 and SGLang digest 506525a5907ea22c9d445afb7c03603959b912de034d86915cf17da814f1a124 for one 32 GB RTX 5090. It uses TP=1, 131,072-token context, concurrency one, FP8 E4M3 KV, CPU media transport, no speculation, eight-image/two-video limits, vllm-hfcache, and the qwen38-radixark-nvfp4-sglang-cache runtime volume.

The image reference records both a repository tag and digest. Preserve the digest; the tag alone is mutable, and availability of that repository/tag is external to this project. This recipe does not include router admission or a promotion contract.

128K qualification · Model dossier

Full tracked recipe

schema = "anvil-serving.serve-recipes/v1"

[[recipe]]
model = "RadixArk/Qwen3.8-27B-NVFP4-RTX5090-128K"
status = "qualification-candidate"
source = "2026-08-17 RTX 5090 128K direct multimodal qualification"
family = "qwen3.8-dense-vl"
total_params_b = 27

[recipe.download]
repo = "RadixArk/Qwen3.8-27B-NVFP4"
revision = "554ebba9b5f1b79dc11246341960360e6ef05ef4"
volume = "vllm-hfcache"
require_complete_cache = true

[recipe.hardware]
gpu = "NVIDIA GeForce RTX 5090"
arch = "sm_120"
vram_total_gb = 32
vram_note = "Single-GPU qualification host; preserve 3 GiB for Windows and driver overhead."

[recipe.serve]
engine = "sglang"
image = "lmsysorg/sglang:qwen38-27b@sha256:506525a5907ea22c9d445afb7c03603959b912de034d86915cf17da814f1a124"
entrypoint = ["python3", "-m", "sglang.launch_server"]
model_flag = "--model-path"
model_path = "/root/.cache/huggingface/hub/models--RadixArk--Qwen3.8-27B-NVFP4/snapshots/554ebba9b5f1b79dc11246341960360e6ef05ef4"
quantization = "ModelOpt NVFP4 W4A4 with FP8 projections, BF16 vision/MTP tensors, and FP8 E4M3 KV cache"
context_tokens = 131072
port = 39038
startup_timeout_seconds = 2400
served_model_name = "qwen38-27b-radixark-nvfp4-sglang-rtx5090-128k-mm"
ipc = "host"
shm_size = "16gb"
named_volumes = [
  "qwen38-radixark-nvfp4-sglang-cache:/root/.cache/sglang",
]
env = [
  "CUDA_DEVICE_ORDER=PCI_BUS_ID",
  "PYTHONHASHSEED=0",
  "SGLANG_USE_CUDA_IPC_TRANSPORT=0",
  "SGLANG_VIT_ENABLE_CUDA_GRAPH=0",
  "SGLANG_MM_BUFFER_SIZE_MB=0",
]
flags = [
  "--served-model-name qwen38-27b-radixark-nvfp4-sglang-rtx5090-128k-mm",
  "--tp-size 1",
  "--mm-feature-transport cpu",
  "--limit-mm-data-per-request '{\"image\":8,\"video\":2}'",
  "--context-length 131072",
  "--max-running-requests 1",
  "--mem-fraction-static 0.85",
  "--kv-cache-dtype fp8_e4m3",
  "--attention-backend flashinfer",
  "--chunked-prefill-size 2048",
  "--disable-radix-cache",
  "--max-mamba-cache-size 1",
  "--reasoning-parser qwen3",
  "--tool-call-parser qwen3_coder",
  "--default-chat-template-kwargs '{\"enable_thinking\":false}'",
  "--host 0.0.0.0",
  "--port 39038",
]
note = "Direct-only RTX 5090 qualification profile. It raises the proven 64K baseline to 128K while retaining concurrency one, FP8 KV, CPU multimodal feature transport, disabled radix caching, one GDN state slot, and no MTP. The 8-image and 2-video ceilings are qualification targets, not inherited capability claims. No route or promotion authority."

[recipe.fit]
suited = ["qualification", "long-context", "multimodal", "vision", "ocr", "video", "tool-use"]
rationale = "The pinned 64K runtime allocated 167,789 KV tokens with 3.73 GB of post-start headroom. A 131,072-token request ceiling preserves roughly 36K KV tokens for output and scheduling while remaining below the measured pool; the exact 120K and multimodal hard gates decide retention."

[recipe.sources]
official = [
  "https://github.com/sgl-project/sglang/blob/70e291b70f5a2833291fff517a00b2f3ff559463/docs/cookbook/autoregressive/Qwen/Qwen3.8-27B.mdx",
  "https://github.com/sgl-project/sglang/blob/70e291b70f5a2833291fff517a00b2f3ff559463/docs/src/snippets/configs/Qwen/qwen3.8-27b.jsx",
]
community = [
  "https://huggingface.co/RadixArk/Qwen3.8-27B-NVFP4/tree/554ebba9b5f1b79dc11246341960360e6ef05ef4",
]
local = [
  "docs/benchmarks/models/qwen38-27b.md",
  "docs/findings/2026-08-17-qwen38-27b-radixark-nvfp4-rtx5090.md",
]

Additional retained families

These remain useful exact sources, but expanding every TOML file here would make the quick-reproduction path harder to scan.

Family	What must travel with it	Recipe sources	Evidence
GLM-5.3-Flash EXL3 K3 + DFlash2 K5, 1M same-model rollback	Purtell-derived digest-pinned image; target and DFlash2 snapshots; vllm-hfcache; glm53-purtell-k3-dflash2-vllm-cache; WSL2 TP=2/DCP=2 controls; DFlash2 noncommercial license boundary	K5 recipe	1M optimization
DeepSeek V4 Flash 0731 Infernal Invocation r18, 1M	Custom digest-pinned voipmonitor/vllm image with embedded wrapper scripts; exact model snapshot; deepseek-v4-0731-r16-hfcache; separate JIT and temporary named volumes; WSL2 TP=2/DCP=1; max-seq-8/batch-4,096; zero offload	DSpark K5 · no-spec control	r18 promotion-era evidence

What this page does not reconstruct

A container recipe does not contain the benchmark corpus, request order, warm/cold state, client clock, measurement instrument, router admission, or promotion decision. Follow the linked finding and raw artifacts before comparing a reproduced number. A healthy response proves neither capacity nor quality, and a configuration copied to different hardware is a new candidate, not a reproduced result.