A copy-paste serving recipe for DeepSeek-V4.1-Flash at EXL3 2bpw on 2× RTX PRO 6000 Blackwell (sm_120): rootless podman over the diffbot pack, eager mode forced by the Engram-on-NVMe path, the generation_config.json that ships official sampling to every client, the reasoning-effort knob that wants strings and not integers, and the measured performance envelope of the result.
A 15s arcade-to-live-action video edit on H3 failed four different ways before it succeeded: cudnn graph crashes on long sequences, a 30s async-output watchdog that throws away finished renders, flashinfer kernels that require tcgen05 MMA my RTX PRO 6000s don’t have, and a finisher script that reported failure after succeeding. The winning recipe: patch the watchdog via bind-mount, CUDNN_ATTN, 540p reference, 1h45m end to end.
Flash-Next is Qwen’s sparse-attention flagship and its official FP8 recipe assumes datacenter GB300. Adapting it to 2x96 GB workstation Blackwell took one legit trick (offloading the 51B-entry PLE n-gram table to pinned host RAM, freeing ~25 GiB per GPU for KV), one config archaeology exercise (YaRN via –hf-overrides because the build has no –rope-scaling flag), and discipline about what not to retry: fp8 KV hard-requires BF16 on QSA and fails differently on vLLM and SGLang, and 786K+ contexts OOM during compile no matter the KV math. Final stack: 512K context, 1.42M-token pool, 146.5 tok/s decode with MTP ns=3, 9-11.2K tok/s prefill, vision and tools validated. The SGLang NVFP4 day-0 lane runs too (after three SM120 gates), but serves one request at 262K slower than the vLLM lane serves two at 512K, so vLLM keeps the port.
H3 ships as two checkpoints: FL2VA (text + first/last frame) and Ref2VA (references including video). Serving both DiTs from one process took three failed hypotheses, a basename-sensitive model resolver, a 66 GB minimal partition download, and one multipart field spelled plural. Now one systemd unit serves text-to-video, keyframe animation, and video editing at 56 GiB per GPU, validated end to end.