Qwen3.8-Flash-Next on DGX Spark — the fastest model on our two-node cluster

What Qwen3.8-Flash-Next is

Qwen3.8-Flash-Next is a model from Alibaba's Qwen team, released on August 26, 2026. Its makers present it as a preview of the Qwen4 architecture per source. The key facts:

• 125 billion parameters in a mixture-of-experts design, of which only 6 billion work on each token. For comparison: GLM 5.3 Flash — 18 billion, DeepSeek V4 Flash — 13 billion. That's where the speed comes from: each token reads the fewest weights from memory.
• Plus 51 billion parameters in an n-gram table and 4 billion in an MTP head — about 180 billion on disk in total.
• Gated DeltaNet and Qwen Sparse Attention; 512 experts, of which 10 plus 1 shared work on each token; 48 layers.
• Multimodal (image, video); a 262,144-token window, up to 1M with YaRN.
• qwen-community-1.0 license — not Apache 2.0 like earlier Qwens (details below).

What Qwen3.8-Flash-Next is made of

parameters per the maker · about 180B on disk in total

125B MoE

51B n-grams

■ MoE experts · ■ n-gram table · ■ MTP head (4B)

active parameters per token — fewer = faster on GB10

Qwen3.8-Flash-Next

6B

DeepSeek V4 Flash

13B

GLM 5.3 Flash

18B

The n-gram table: 51B parameters, of which a token needs 16 rows

The most interesting part of the design. On its second layer the model has a table of 20 million bigrams and trigrams (PLE). For each token it needs only 16 rows of it — so the table doesn't have to sit in GPU memory. It can live in regular CPU memory, or even on an NVMe drive. That's what lets the model fit in less memory than its parameter count suggests.

How much of the table one token needs

illustration, numbers per the maker

20,000,000

rows in the table (bigrams and trigrams)

per token

16

rows — readable from RAM or an NVMe drive

Qwen3.8-Flash-Next weights: NVFP4, FP8 and GGUF

Weight size vs DGX Spark memory

sizes as reported by the sources · │ 128 GB — one unit · │ 256 GB — two

BF16

335 GiB

FP8

173 GiB

NVFP4 · ours

124 GB

NVIDIA's model card officially lists only B200 and B300 GPUs — that the NVFP4 checkpoint runs on GB10 is known from independent deployments, ours included. According to the tonyd2wild recipe author, the model also runs on a single DGX Spark (n-gram table read from disk) at 43.9 tok/s per source. We haven't measured that setup.

How we run Qwen3.8-Flash-Next on DGX Spark with vLLM

Same cluster as for DeepSeek V4 Flash and GLM 5.3 Flash: two units linked with a QSFP56 200 Gb/s cable, the model split across both (tensor parallel = 2), with a chat app, an API gateway and agents alongside. Same test method too. Hardware details: DGX Spark cluster.

Unlike GLM, Qwen runs on stock vLLM with no patches — the September 14 nightly was enough. The official launch-day image was too old: NVIDIA's checkpoint and MTP needed later vLLM fixes.

Qwen3.8-Flash-Next benchmark on DGX Spark

Generation speed

tok/s, median of 6 · measured by us September 14, 2026 · vLLM nightly, NVFP4, MTP 3 · 2× DGX Spark, apps alongside

JSON

63.3first token after 142 ms

code

57.4first token after 161 ms

prose

37.9first token after 152 ms

6 conversations, each

34.1150.7 tok/s total · first token after 250 ms

• KV pool: 700,981 tokens (2.67× the window). After the run, 23.8 / 26.1 GB stays free — the most headroom of the three models.
• Startup: about 8.5 minutes.
• Agreement with the recipe. The tonyd2wild recipe author measured 40 prompts in 8 categories: prose 37.2 vs our 37.9, code 56.0 vs 57.4, JSON 61.9 vs 63.3. Very close, despite our apps running alongside — a good sign the setup is reproducible.

According to the same source (we didn't measure these): one unit 43.9 tok/s; a long-context profile on two units (table on NVMe) 35.8 tok/s with a pool of about 5.87M tokens; and without MTP and CUDA graphs only 15.4 tok/s. So on GB10, MTP clearly helps — the opposite of the warning for H100 GPUs per source.

Newer vLLM isn't faster: vLLM 0.31 was 30% slower

On October 8 we checked, with the same method, whether to move from the September 14 nightly to the vLLM 0.31 release. The result was a surprise: on this model and GB10, vLLM 0.31 is about 30% slower. We haven't found the cause — it's a regression, not a settings issue.

vLLM nightly (Sept 14) vs vLLM 0.31

tok/s · measured by us October 8, 2026, same method · ■ nightly, KV BF16 ■ 0.31, KV BF16 ■ 0.31, KV FP8

prose

nightly

38.7

0.31 BF16

27.5

0.31 FP8

27.3

code

nightly

58.9

0.31 BF16

42.4

0.31 FP8

39.1

6 conversations at once, total

nightly

144

0.31 BF16

92

0.31 FP8

90

Measured by us October 8, 2026. The nightly results in this series differ slightly from the September 14 run — that's variation between series, which is why we only compare columns from the same day.

One thing in 0.31 is genuinely good: an FP8 KV cache “for free”. Same speed as BF16, and the pool grows 1.64× (not 2× — some layers don't benefit), with long-context quality intact (needle 6/6). We still stayed on the September 14 nightly: 30% of speed is worth more than a bigger pool.

Images and long context

On September 15 we sent 10 images in one message. All were described correctly, including small text. Without thinking it took 7.9 s (3,063 input tokens), with thinking 14 s measured by us. We checked long context on vLLM 0.31 with an FP8 KV cache: the needle was found at 112K and 225K tokens in all 6 runs.

Silent failures

• torch.compile + 4,096-token chunks = 8–15 tok/s instead of 53. According to the recipe author, this combination also creates a second copy of the n-gram table (about 24 GB per unit) — for him it ended with restarting three DGX Spark units per source. That's why we keep torch.compile off.
• Every model family has its own thinking switch. Qwen reads enable_thinking, DeepSeek reads thinking. Our API gateway sent only one of them, so with Qwen the conversation titles came back empty (the model “thought” through the whole 40-token budget). Now the gateway sends both.
• gpu-memory-utilization 0.75 tempts with a bigger pool (1.12M tokens), but prose speed then jumped between 28 and 40 tok/s. We stayed at 0.70.
• A newer engine release can be slower — measure before you update vLLM, don't assume.

Pitfalls shared by all GB10 machines — freezing when memory runs out, and the wrong RDMA device — are on the DeepSeek V4 Flash page.

License: what a company may do

Qwen3.8-Flash-Next is under the qwen-community-1.0 license (the NVFP4 checkpoint additionally under the NVIDIA Open Model License). That's no longer Apache 2.0 like earlier Qwens, and for companies it matters:

• Internal use is exempt — when neither the model nor its outputs reach third parties.
• “Model as a Service” (offering inference to other companies) or an “AI Work Assistant” product requires a separate license from Qwen before commercial use — with no revenue threshold.
• The obligation to display the model's name only applies above 100 million monthly active users or $20 million per month.

In practice: for work inside a company, Qwen is an excellent choice. If the model is going to answer your customers, pick DeepSeek V4 Flash or GLM 5.3 Flash under MIT — or talk to Qwen first.

Qwen3.8-Flash-Next compared with DeepSeek V4 Flash and GLM 5.3 Flash

On the same cluster, Qwen starts answering fastest (142–161 ms), delivers 150.7 tok/s in total with six conversations at once, and leaves the most free memory (24–26 GB per unit). It's the only one that runs on stock vLLM with no patches. DeepSeek V4 Flash writes prose faster in daily use (44–53 tok/s), reads long documents fastest and is MIT. GLM 5.3 Flash on TensorFold writes code and JSON fastest and has a 1M-token window, but only for non-commercial use. The full side-by-side comparison, with charts and seven lessons from running them: Best LLM for 2× DGX Spark.

Qwen3.8-Flash-Next FAQ

Does Qwen3.8-Flash-Next run on one DGX Spark?

According to the tonyd2wild recipe author, yes: on one unit, with the n-gram table read from disk, it reaches 43.9 tok/s. The NVFP4 weights are 124 GB including the table. We run it on two units and haven't measured the single-unit setup.

Can Qwen3.8-Flash-Next be used commercially?

Internally, yes — the qwen-community-1.0 license exempts use where neither the model nor its outputs reach third parties. Offering inference to other companies (“Model as a Service”) or an “AI Work Assistant” product requires a separate license from Qwen before commercial use, regardless of revenue.

Does Qwen3.8-Flash-Next understand images locally?

Yes. In our test, 10 images sent in one message were all described correctly, including small text: 7.9 s without thinking and 14 s with thinking, on a 2× DGX Spark cluster, without sending anything to the cloud.

Why turn off torch.compile for Qwen3.8-Flash-Next?

According to the recipe author, torch.compile combined with 4,096-token chunks drops speed to 8–15 tok/s instead of 53 and creates a second copy of the n-gram table (about 24 GB per unit) — for him it ended with restarting three DGX Spark units. We keep torch.compile off.

Is it worth updating vLLM for Qwen3.8-Flash-Next?

Not always on GB10. In our October 8, 2026 measurement, vLLM 0.31 was about 30% slower than the September 14 nightly (prose 27.5 vs 38.7 tok/s). The upside of 0.31 is FP8 KV cache support: the same speed and a 1.12M-token pool instead of 686K. We stayed on the nightly.

Sources

• Qwen, Qwen3.8-Flash-Next model card and the qwen-community-1.0 license.
• NVIDIA, Qwen3.8-Flash-Next-NVFP4 — size, n-gram table, officially supported GPUs.
• tonyd2wild, Qwen3.8-Flash-Next NVFP4 on DGX Spark (recipe) — 40-prompt results, single unit, long-context profile, the torch.compile pitfall.
• vLLM, Qwen3.8-Flash-Next recipe.
• NVIDIA Developer Forums, Qwen 3.8 Flash Next on a single node with vLLM.
• MarkTechPost, Qwen3.8-Flash-Next release (August 26, 2026).
• Measurements and tests: our own, ailone's 2× DGX Spark / ASUS GX10 cluster, September 14–15 and October 8, 2026.

As of October 11, 2026. vLLM versions change every week — measure speed on your own hardware before you update. Measurements are medians from one series, with apps running alongside.