Two Sparks, Qwen Flash-Next, and the Config I Actually Kept

I switched my local AI stack back to Qwen, tuned it across two NVIDIA DGX Sparks, and put the configuration on GitHub. The useful numbers are about 3,000 input tokens per second, 76.5 output tokens per second in the repeatable single-request coding speed test, and 176.9 output tokens per second total with four requests running together.

Those are different measurements. Keeping them separate matters more than picking the biggest number for a headline.

The recipe, launch scripts, benchmark scripts and raw results are here. This is my deployment of Local Inference Lab’s work, with the settings I tested and kept. It is not an official LIL release.

What is running

Both Sparks run one model together using tensor parallelism, or TP2. That means each machine holds part of the model and they communicate while generating an answer. They are not two independent chat servers.

The runtime is an ARM64 build of LIL’s October 16.2 Karmic Kraken recipe. The model is Qwen3.8 Flash-Next NVFP4/MXFP8 CSF-QAD, pinned to revision dc5b8ab2.

The settings that define this deployment:

• Maximum context of 262,144 tokens, with up to 16 sequences.
• GPU memory utilization set to 0.55, leaving room for other work on these machines.
• FP8 main KV cache and B12X loading and compute backends.
• MTP speculative decoding: four draft tokens for one request, three for batches of two through sixteen.
• One-shot RoCEnante enabled, with B12X autotuning on.
• Medium reasoning by default. A client can request a different effort level.

The full environment and flags are in the repository. The pinned LIL build source and checkpoint are linked there, alongside the component build receipt.

The numbers I can reproduce

Test	Result
Fixed coding speed test, one request	76.55 tok/s decode
Fixed coding speed test, four concurrent requests	176.88 tok/s aggregate, end-to-end
Uncached 8,192-token input	3,126 tok/s effective prefill; 2.62 s to first token
Uncached 32,768-token input	2,980 tok/s effective prefill; 11.00 s to first token
Hotel Lights, medium reasoning, four concurrent requests	8/8 correct; 54.3 tok/s per request

Open the full benchmark chart for the intermediate configurations and comparisons.

The coding speed test asks for a Python LRU cache implementation, forces 1,024 output tokens, and runs three repetitions at concurrency one and four. Temperature is zero and reasoning is medium. It is a throughput test, not a score for whether the generated program works. The single-request recheck samples ranged from about 70.9 to 83.1 tokens per second.

Prefill is the model reading the input before answering. Those figures use exact input token counts, three samples per length, and zero cached prompt tokens. I divide input length by time to first token, so the rate includes request overhead and the first output token. It is an effective serving measurement, not isolated GPU kernel speed.

The 262K context setting is capacity. I have not run a full long-context decode matrix, and I am not claiming the short-prompt speed at every context length.

Hotel Lights is a separate reasoning check. The tuned configuration got eight out of eight with no errors, stalls or truncations in that run. It is useful evidence for this workload, but eight answers do not establish general model quality.

Why I kept one-shot RoCEnante

We tried two-shot as well. There was a trap: an early integration did not actually activate the intended path. Those results do not count as a two-shot comparison.

After fixing the initialization and confirming activation in the logs, the two-shot trial measured 74.7 tok/s for one request and 178.7 tok/s aggregate at four requests. Effective prefill was 2,959 tok/s at 8K and 2,974 tok/s at 32K.

That is close to the retained one-shot configuration. It was not a clear overall improvement, so I kept one-shot. A slightly higher aggregate number in one small run was not enough reason to replace it.

We also tried drafting five tokens with MTP. That configuration did not make the cut. The chart includes it because rejected settings are part of the result, too.

What OpenCode2 sees

After the switch, I gave OpenCode2 a large brief for a multiplayer browser game and watched the server while it worked. During sustained generation, some ten-second server windows showed 90–99 tok/s. Recent completed client turns worked out to roughly 78–85 tok/s, including reasoning and request overhead.

That is a live observation, not a replacement for the fixed benchmark. The game session used high reasoning, its context changed as it read and wrote files, and cached prefixes reduced repeated input work. It had started writing files; that observation does not prove the game was complete or correct.

This is why I wanted the raw results published. A burst on a dashboard, a full agent turn, and four users sharing a server answer different questions.

The rest of the stack

The local generation routes for Captain Jack, my search monitors, and the daily fishing and social pipelines now point to Qwen. That does not mean every stage runs locally. Existing review steps still use Claude, and OMG’s code builder retains its Sonnet default.

The migration also needed more than changing a model dropdown. Scheduled jobs can pin their own model, and clients can validate the model name returned by the API. We checked those paths, corrected two leftover Nemotron job overrides, and tested structured responses and image input. Compatibility names remain available for older clients, while qwen3.8-flash-next is the canonical model name.

Can ComfyUI stay up?

Yes, with limits. ComfyUI remains available on one Spark, with about 29 GiB of memory available at idle in this deployment. Earlier measurements showed Qwen-Image 2.1 INT8 with Turbo8 coexisting with the coding model at 1024×1024, one image at a time. That image setup used about 18.7 GB and slowed coding while rendering.

I have not repeated the render benchmark on this newly promoted build. Large installed video workflows still pause the coding model. A memory-utilization flag is not a promise that any second model will fit.

Try the recipe

The GitHub repository includes parameterized rank scripts, Docker Compose, the chat template, source pins, and the two speed probes. It also explains the build limits and the libaio clone workaround we needed.

The image was built locally; the repository does not distribute a ready-made image or weights. The portable configuration has been checked, but I have not cold-started a second cluster from that export or repeated its simplified clean-build path. The published measurements belong to the running deployment described here.

Credit belongs to Local Inference Lab and the upstream B12X/RoCEnante contributors for the model packaging, runtime, kernels and transport. My contribution here is the configuration, the comparisons, and a record of what actually ran on these two machines.