MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks

DeepSeek-V4.1-Flash on 3x and 4x NVIDIA DGX Sparks by Mia'a AI Lab Serve deepseek-ai/DeepSeek-V4.1-Flash-1017 with SGLang on three DGX Spark boxes (GB10, SM121, 121.7 GiB unified memory each) over a ConnectX-7 RoCE triangle: native MXFP4 experts, FP8 dense weights, NVMe-resident Engram tables, DSpark speculative decoding, tool calling and vision, behind one OpenAI-compatible endpoint. On disk the checkpoint is 476 GiB. After moving the two Engram n-gram tables (189 GiB) to NVMe, the GPU-resident part is 305 GiB of MXFP4 experts plus FP8 dense weights: Host RAM is GPU memory on a Spark. Everything that pins host memory (NCCL buffers, row caches, tokenizer processes) competes with the model, and a head node that runs out of memory stalls all three ranks because TP is synchronous. Most of what this repo does beyond the upstream recipe is about that. Never use OFFLOADMODE=ram here. The Engram tables would evict the model. vllm-fn-nfs exporter…