(APIServer pid=1) INFO 09-24 04:57:08 [api_utils.py:347] 
(APIServer pid=1) INFO 09-24 04:57:08 [api_utils.py:347]        █     █     █▄   ▄█
(APIServer pid=1) INFO 09-24 04:57:08 [api_utils.py:347]  ▄▄ ▄█ █     █     █ ▀▄▀ █  version 0.29.0
(APIServer pid=1) INFO 09-24 04:57:08 [api_utils.py:347]   █▄█▀ █     █     █     █  model   /container-cache/huggingface/hub/models--bottlecapai--ThinkingCap-Qwen3.8-27B-NVFP4A4-AWQ/snapshots/f8fe157f207a13f977bc3d620ce10a3e9ba5ab11
(APIServer pid=1) INFO 09-24 04:57:08 [api_utils.py:347]    ▀▀  ▀▀▀▀▀ ▀▀▀▀▀ ▀     ▀
(APIServer pid=1) INFO 09-24 04:57:08 [api_utils.py:347] 
(APIServer pid=1) INFO 09-24 04:57:08 [api_utils.py:286] non-default args: {'enable_auto_tool_choice': True, 'tool_call_parser': 'qwen3_xml', 'host': '0.0.0.0', 'port': 39283, 'model': '/container-cache/huggingface/hub/models--bottlecapai--ThinkingCap-Qwen3.8-27B-NVFP4A4-AWQ/snapshots/f8fe157f207a13f977bc3d620ce10a3e9ba5ab11', 'max_model_len': 131072, 'served_model_name': ['thinkingcap-awq-5090'], 'generation_config': 'vllm', 'attention_backend': 'TRITON_ATTN', 'reasoning_parser': 'qwen3', 'gpu_memory_utilization': 0.93, 'kv_cache_dtype': 'bfloat16', 'enable_prefix_caching': False, 'offload_backend': 'uva', 'cpu_offload_gb': 3.0, 'limit_mm_per_prompt': {'image': 2, 'video': 0}, 'mm_processor_kwargs': {'size': {'longest_edge': 1048576, 'shortest_edge': 65536}}, 'max_num_batched_tokens': 1024, 'max_num_seqs': 1, 'speculative_config': {'method': 'mtp', 'num_speculative_tokens': 3, 'attention_backend': 'TRITON_ATTN'}}
(APIServer pid=1) INFO 09-24 04:57:08 [model.py:684] Resolved architecture: Qwen3_5ForConditionalGeneration
(APIServer pid=1) INFO 09-24 04:57:08 [model.py:2021] Using max model len 131072
(APIServer pid=1) INFO 09-24 04:57:15 [model.py:684] Resolved architecture: Qwen3_5MTP
(APIServer pid=1) INFO 09-24 04:57:15 [model.py:2021] Using max model len 262144
(APIServer pid=1) INFO 09-24 04:57:15 [model.py:887] Disabled mm_prefix attention mode because multimodal inputs are configuration-disabled. Attention backends without mm_prefix support may now be selected.
(APIServer pid=1) WARNING 09-24 04:57:15 [speculative.py:1318] Enabling num_speculative_tokens > 1 will run multiple times of forward on same MTP layer,which may result in lower acceptance rate
(APIServer pid=1) INFO 09-24 04:57:15 [speculative.py:1586] Overriding draft model max model len from 262144 to 131072
(APIServer pid=1) INFO 09-24 04:57:15 [arg_utils.py:2404] Hybrid model with EAGLE speculative decoding: defaulting prefix_cache_retention_interval to dense checkpointing.
(APIServer pid=1) INFO 09-24 04:57:15 [scheduler.py:277] Chunked prefill is enabled with max_num_batched_tokens=1024.
(APIServer pid=1) INFO 09-24 04:57:15 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=1) WARNING 09-24 04:57:15 [vllm.py:1924] max_num_scheduled_tokens is set to 1024 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens.
(APIServer pid=1) WARNING 09-24 04:57:15 [cuda.py:338] --cpu-offload-gb is enabled with CUDA graphs on WSL2. This combination requires pinned (page-locked) memory allocations. WARNING: Windows (WDDM) enforces a hard system-wide cap of roughly 50%% of physical RAM on pinned memory shared across ALL processes by default (limit can changed via %OPERATOR_HOME%\.wslconfig). Excessive use of page-locked memory can prevent Windows from reclaiming memory under load, which can cause the entire host OS to become unresponsive and may require a hard reboot to recover. Proceed at your own risk. To raise the WSL2 VM memory ceiling, increase the `memory` setting in %OPERATOR_HOME%\.wslconfig and run `wsl --shutdown`.
(EngineCore pid=115) INFO 09-24 04:57:32 [core.py:123] Initializing a V1 LLM engine (v0.29.0) with config: model='/container-cache/huggingface/hub/models--bottlecapai--ThinkingCap-Qwen3.8-27B-NVFP4A4-AWQ/snapshots/f8fe157f207a13f977bc3d620ce10a3e9ba5ab11', speculative_config=SpeculativeConfig(method='mtp', model='/container-cache/huggingface/hub/models--bottlecapai--ThinkingCap-Qwen3.8-27B-NVFP4A4-AWQ/snapshots/f8fe157f207a13f977bc3d620ce10a3e9ba5ab11', num_spec_tokens=3), tokenizer='/container-cache/huggingface/hub/models--bottlecapai--ThinkingCap-Qwen3.8-27B-NVFP4A4-AWQ/snapshots/f8fe157f207a13f977bc3d620ce10a3e9ba5ab11', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=131072, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=compressed-tensors, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=bfloat16, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='qwen3', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=thinkingcap-awq-5090, enable_prefix_caching=False, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_compute_ple_ngram_ids', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [1024], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 8, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, enable_jit_warmup=True, enable_bf16x3_router_gemm=False, moe_backend='auto', linear_backend='auto')
(EngineCore pid=115) INFO 09-24 04:57:33 [parallel_state.py:1775] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_bd72f405568e44c0986efa583bc8079b backend=nccl
(EngineCore pid=115) INFO 09-24 04:57:33 [parallel_state.py:2119] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=115) INFO 09-24 04:57:33 [gpu_worker.py:429] Using V2 Model Runner
(EngineCore pid=115) INFO 09-24 04:57:35 [base.py:138] Offloader set to UVAOffloader
(EngineCore pid=115) INFO 09-24 04:57:35 [model_runner.py:382] Loading model from scratch...
(EngineCore pid=115) INFO 09-24 04:57:41 [cuda.py:551] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=115) INFO 09-24 04:57:41 [mm_encoder_attention.py:372] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=115) INFO 09-24 04:57:43 [uva.py:65] Total CPU offloaded parameters: 0.86
(EngineCore pid=115) INFO 09-24 04:57:43 [__init__.py:726] Selected CutlassFP8ScaledMMLinearKernel for CompressedTensorsW8A8Fp8
(EngineCore pid=115) INFO 09-24 04:57:43 [deep_gemm.py:186] deep_gemm not found in site-packages, trying vendored vllm.third_party.deep_gemm
(EngineCore pid=115) INFO 09-24 04:57:43 [deep_gemm.py:213] DeepGEMM PDL enabled on vllm.third_party.deep_gemm.
(EngineCore pid=115) INFO 09-24 04:57:43 [deep_gemm.py:131] DeepGEMM E8M0 enabled on current platform.
(EngineCore pid=115) INFO 09-24 04:57:43 [qwen_gdn_linear_attn.py:167] Using Triton/FLA GDN prefill kernel (requested=auto, head_k_dim=128).
(EngineCore pid=115) INFO 09-24 04:57:43 [qwen_gdn_linear_attn.py:519] GDN decode kernel: cuda
(EngineCore pid=115) INFO 09-24 04:57:43 [__init__.py:1106] Using FlashInferCutlassNvFp4LinearKernel for NVFP4 GEMM
(EngineCore pid=115) INFO 09-24 04:57:44 [cuda.py:432] Using AttentionBackendEnum.TRITON_ATTN backend.
(EngineCore pid=115) INFO 09-24 04:57:46 [uva.py:65] Total CPU offloaded parameters: 3.02
(EngineCore pid=115) INFO 09-24 04:57:46 [weight_utils.py:863] Filesystem type for checkpoints: EXT4. Checkpoint size: 21.81 GiB. Available RAM: 10.75 GiB.
(EngineCore pid=115) INFO 09-24 04:57:46 [weight_utils.py:893] Auto-prefetch is disabled because the filesystem (EXT4) is not a recognized network FS (NFS/Lustre) and the checkpoint size (21.81 GiB) exceeds 90% of available RAM (10.75 GiB).
(EngineCore pid=115) INFO 09-24 04:58:01 [default_loader.py:430] Loading weights took 15.23 seconds
(EngineCore pid=115) INFO 09-24 04:58:04 [weight_utils.py:863] Filesystem type for checkpoints: EXT4. Checkpoint size: 21.81 GiB. Available RAM: 10.56 GiB.
(EngineCore pid=115) INFO 09-24 04:58:04 [weight_utils.py:893] Auto-prefetch is disabled because the filesystem (EXT4) is not a recognized network FS (NFS/Lustre) and the checkpoint size (21.81 GiB) exceeds 90% of available RAM (10.56 GiB).
(EngineCore pid=115) INFO 09-24 04:58:06 [default_loader.py:430] Loading weights took 2.46 seconds
(EngineCore pid=115) INFO 09-24 04:58:07 [model_runner.py:404] Model loading took 19.11 GiB memory and 32.644009 seconds
(EngineCore pid=115) INFO 09-24 04:58:07 [topk_topp_sampler.py:62] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=115) INFO 09-24 04:58:07 [interface.py:918] Setting attention block size to 800 tokens to ensure that attention page size is >= mamba page size.
(EngineCore pid=115) INFO 09-24 04:58:07 [interface.py:942] Padding mamba page size by 0.25% to ensure that mamba page size and attention page size are exactly equal.
(EngineCore pid=115) INFO 09-24 04:58:07 [utils.py:306] Using LBNHC KV cache layout.
(EngineCore pid=115) INFO 09-24 04:58:13 [encoder_runner.py:131] Encoder cache will be initialized with a budget of 1024 tokens, and profiled with 1 image items of the maximum feature size.
(EngineCore pid=115) INFO 09-24 04:58:46 [backends.py:1094] Using cache directory: /container-cache/vllm/torch_compile_cache/bbe52cf0c2/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=115) INFO 09-24 04:58:46 [backends.py:1155] Dynamo bytecode transform time: 13.59 s
(EngineCore pid=115) INFO 09-24 04:59:15 [backends.py:393] Compiling a graph for compile range (1, 1024) takes 28.25 s
(EngineCore pid=115) INFO 09-24 04:59:23 [backends.py:920] collected artifacts: 65 entries, 23 artifacts, 55643919 bytes total
(EngineCore pid=115) INFO 09-24 04:59:23 [decorators.py:719] saved AOT compiled function to /container-cache/vllm/torch_compile_cache/torch_aot_compile/db2d17a434bf15f13746ce84eb65580d9d4a72504c9ccc8984d26521019dedef/rank_0_0/model
(EngineCore pid=115) INFO 09-24 04:59:23 [monitor.py:53] torch.compile took 50.47 s in total
(EngineCore pid=115) INFO 09-24 05:00:10 [monitor.py:81] Initial profiling/warmup run took 47.29 s
(EngineCore pid=115) INFO 09-24 05:00:10 [backends.py:1094] Using cache directory: /container-cache/vllm/torch_compile_cache/bbe52cf0c2/rank_0_0/eagle_head for vLLM's torch.compile
(EngineCore pid=115) INFO 09-24 05:00:10 [backends.py:1155] Dynamo bytecode transform time: 0.23 s
(EngineCore pid=115) INFO 09-24 05:00:13 [backends.py:393] Compiling a graph for compile range (1, 1024) takes 2.93 s
(EngineCore pid=115) INFO 09-24 05:00:14 [backends.py:920] collected artifacts: 2 entries, 2 artifacts, 3383410 bytes total
(EngineCore pid=115) INFO 09-24 05:00:14 [decorators.py:719] saved AOT compiled function to /container-cache/vllm/torch_compile_cache/torch_aot_compile/cfcef722a1dbad2d442780668298e05def95289f2cd128bf46f9f1e82ee956c5/rank_0_0/model
(EngineCore pid=115) INFO 09-24 05:00:14 [monitor.py:53] torch.compile took 3.31 s in total
(EngineCore pid=115) INFO 09-24 05:00:14 [monitor.py:81] Initial profiling/warmup run took 0.00 s
(EngineCore pid=115) WARNING 09-24 05:00:16 [kv_cache_utils.py:1326] Add 3 padding layers, may waste at most 6.25% KV cache memory
(EngineCore pid=115) INFO 09-24 05:00:20 [speculator.py:151] Capturing model for speculator...
(EngineCore pid=115) INFO 09-24 05:00:20 [model_runner.py:960] Graph capturing finished in 4 secs, took 0.14 GiB
(EngineCore pid=115) INFO 09-24 05:00:21 [gpu_worker.py:625] Available KV cache memory: 9.7 GiB
(EngineCore pid=115) INFO 09-24 05:00:21 [gpu_worker.py:640] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9300 is equivalent to --gpu-memory-utilization=0.9256 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9344. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=115) WARNING 09-24 05:00:21 [kv_cache_utils.py:1326] Add 3 padding layers, may waste at most 6.25% KV cache memory
(EngineCore pid=115) INFO 09-24 05:00:21 [kv_cache_utils.py:2032] GPU KV cache size: 138,519 tokens, Maximum concurrency for 131,072 tokens per request: 1.06x
(EngineCore pid=115) INFO 09-24 05:00:21 [kernel_warmup.py:124] JIT kernel warmup starting.
(EngineCore pid=115) INFO 09-24 05:00:21 [kernel_warmup.py:134] JIT kernel warmup finished in 0.00s.
(EngineCore pid=115) INFO 09-24 05:00:21 [qwen_triton_warmup.py:270] Warming up Qwen Triton kernels for model_type=qwen3_5_text.
(EngineCore pid=115) INFO 09-24 05:00:22 [kernel_warmup.py:309] Using FlashInfer autotune cache file: /container-cache/vllm/flashinfer_autotune_cache/0.6.18/120f/4eb0e305034bc8582b36f84cdb988fb7ee3e83381b1d1a77d69e291937b5b808/autotune_configs.json
(EngineCore pid=115) INFO 09-24 05:00:22 [kernel_warmup.py:266] Running FlashInfer autotune with 1024 tokens.
(EngineCore pid=115) INFO 09-24 05:01:13 [speculator.py:151] Capturing model for speculator...
(EngineCore pid=115) INFO 09-24 05:01:13 [model_runner.py:960] Graph capturing finished in 2 secs, took 0.10 GiB
(EngineCore pid=115) INFO 09-24 05:01:13 [gpu_worker.py:797] CUDA graph pool memory: 0.1 GiB (actual), 0.14 GiB (estimated), difference: 0.04 GiB (36.5%).
(EngineCore pid=115) INFO 09-24 05:01:13 [gpu_worker.py:860] Free memory on device (30.2/31.82 GiB) on startup. Desired GPU memory utilization is (0.93, 29.59 GiB). Actual usage is 19.47 GiB for consumed memory (weights + non-torch), 0.42 GiB for peak activation, and 0.1 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=10146379695` (9.45 GiB) to fit into requested memory, or `--kv-cache-memory=10805014528` (10.06 GiB) to fully utilize gpu memory. Current kv cache memory in use is 9.7 GiB.
(EngineCore pid=115) INFO 09-24 05:01:17 [jit_monitor.py:85] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=115) INFO 09-24 05:01:18 [torch_utils.py:277] Reducing Torch threads from 8 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=115) INFO 09-24 05:01:18 [core.py:361] init engine (profile, create kv cache, warmup model) took 190.23 s (compilation: 53.78 s)
(EngineCore pid=115) INFO 09-24 05:01:18 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(EngineCore pid=115) WARNING 09-24 05:01:18 [vllm.py:1924] max_num_scheduled_tokens is set to 1024 based on the speculative decoding settings. This may lead to suboptimal performance. Consider increasing max_num_batched_tokens to accommodate the additional draft token slots, or decrease num_speculative_tokens.
(EngineCore pid=115) WARNING 09-24 05:01:18 [cuda.py:338] --cpu-offload-gb is enabled with CUDA graphs on WSL2. This combination requires pinned (page-locked) memory allocations. WARNING: Windows (WDDM) enforces a hard system-wide cap of roughly 50%% of physical RAM on pinned memory shared across ALL processes by default (limit can changed via %OPERATOR_HOME%\.wslconfig). Excessive use of page-locked memory can prevent Windows from reclaiming memory under load, which can cause the entire host OS to become unresponsive and may require a hard reboot to recover. Proceed at your own risk. To raise the WSL2 VM memory ceiling, increase the `memory` setting in %OPERATOR_HOME%\.wslconfig and run `wsl --shutdown`.
(APIServer pid=1) INFO 09-24 05:01:19 [entry.py:135] Supported tasks: ['generate']
(APIServer pid=1) INFO 09-24 05:01:19 [parser_manager.py:36] "auto" tool choice has been enabled.
(APIServer pid=1) INFO 09-24 05:01:22 [hf.py:547] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this.
(APIServer pid=1) INFO 09-24 05:01:36 [base.py:244] Multi-modal warmup completed in 13.366s
(APIServer pid=1) INFO 09-24 05:01:36 [base.py:244] Readonly multi-modal warmup completed in 0.015s
(APIServer pid=1) INFO 09-24 05:01:36 [entry.py:139] Starting vLLM server on http://0.0.0.0:39283
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:61] Available routes are:
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /openapi.json, Methods: GET, HEAD
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /docs, Methods: GET, HEAD
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /docs/oauth2-redirect, Methods: GET, HEAD
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /redoc, Methods: GET, HEAD
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /load, Methods: GET
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /version, Methods: GET
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /health, Methods: GET
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /metrics, Methods: GET
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /tokenize, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /detokenize, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/models, Methods: GET
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /ping, Methods: GET
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /ping, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /invocations, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/chat/completions, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/chat/completions/batch, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/responses, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/responses/{response_id}, Methods: GET
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/responses/{response_id}/cancel, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/completions, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/messages, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/messages/count_tokens, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /generative_scoring, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /scale_elastic_ep, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /is_scaling_elastic_ep, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/chat/completions/render, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/messages/render, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/completions/render, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/chat/completions/derender, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /v1/completions/derender, Methods: POST
(APIServer pid=1) INFO 09-24 05:01:36 [launcher.py:70] Route: /inference/v1/generate, Methods: POST
(APIServer pid=1) INFO:     192.0.2.10:52714 - "GET /health HTTP/1.1" 200 OK