DeepSeek V4 Flash on two DGX Sparks — how we run it locally and what it scores

What DeepSeek V4 Flash is

DeepSeek V4 Flash is a language model from the Chinese company DeepSeek. The preview came out on April 24, 2026, and on July 31, 2026 DeepSeek published a refreshed 0731 variant per source. The key facts:

• 284 billion parameters in a mixture-of-experts (MoE) design, of which only 13 billion work on each token. That's why it's fast for its size — each token reads only part of the weights from memory.
• A context window of up to 1 million tokens, natively.
• MIT license — commercial use allowed.
• DSpark — a speculative decoding module attached to the checkpoint. vLLM supports it from version 0.25.0.

Here's how DSpark works: a small module proposes a block of several tokens at once, and the large model checks the whole block in a single pass, keeping the longest run that matches what it would have written itself. When the guess is right, the model gets several tokens for the price of one pass. According to DeepLearning.AI, in DeepSeek's own production DSpark made V4 Flash generate 60–85% faster than the previous module (MTP-1) per source.

How DSpark works

Illustration of one speculative decoding step with 5 tokens ahead.

1 · DSpark proposes

the contract takes effect today

A small module guesses the next 5 tokens at once.

2 · the large model checks

DeepSeek V4 Flash scores the whole block in one pass instead of five.

Cost: almost the same as generating a single token.

3 · the matching start is kept

the ✓ contract ✓ takes ✓ effect today

3 tokens for one pass. The share of tokens kept is the “acceptance rate”.

A note on names: in September 2026 a larger DeepSeek-V4.1-Flash (552 billion backbone parameters, also MIT) appeared in DeepSeek's V4 collection on Hugging Face per source. We haven't run it — this page is about V4 Flash.

DeepSeek V4 Flash requirements: one DGX Spark or two?

Short answer: the version we use needs two DGX Spark units. Only a heavily compressed GGUF fits on one — according to its authors, since we haven't run it ourselves.

DeepSeek V4 Flash GGUF vs our setup

Unsloth publishes DeepSeek-V4-Flash-GGUF at several compression levels and points to llama.cpp built from the main repository to run it. According to Unsloth, DSpark needs about 10 GB more on that route, so on a 128 GB machine it recommends IQ3_XXS with the Q8_0 drafter per source.

There are two differences from our setup: 3 bits per weight instead of FP8, and llama.cpp instead of vLLM — a different engine with different behavior under several concurrent conversations and different tooling. We haven't measured GGUF speed or quality, so we don't compare it with our numbers. With one DGX Spark, that's the route to try; with two, the fuller vLLM setup below.

How we run DeepSeek V4 Flash locally on 2× DGX Spark with vLLM

The cluster is two NVIDIA DGX Spark / ASUS Ascent GX10 units with the GB10 chip, each with 128 GB of unified memory (the system sees 121.6 GB), 256 GB in total. The model is split across both units (tensor parallel = 2). The cluster runs one large model at a time, switched from a dashboard in a few minutes. All apps call one shared model name, so switching models needs no changes in the apps. How the units are linked and how to scale the cluster is on our DGX Spark cluster page.

2× DGX Spark cluster running DeepSeek V4 Flash

One model split across two units. State during our measurements (August 2026).

UNIT 1 · GB10 · 128 GB

79.2 GiB — half of DeepSeek V4 Flash's weights

KV cache for conversations

about 11 GB — chat, API gateway, agents

QSFP56 · 200 Gb/s

RDMA (RoCE) · NCCL · TP=2

UNIT 2 · GB10 · 128 GB

79.2 GiB — the other half of the weights

KV cache for conversations

memory headroom ≥ about 10 GB

We based the setup on public community recipes: MiaAI-Lab's recipe for 2× DGX Spark and Anemll's vLLM image with DSpark support (links in the sources). We don't publish our scripts or network details — the recipes have everything you need to start. The key settings:

How we measure

• Median of 6 runs after 3 warm-up requests — never right after startup, because the first requests compile kernels.
• Temperature 0.6, 400–500-token answers, separately for prose, code and JSON.
• Single runs vary by up to 42% with an identical setup (for example 37.9 and 65.3 tok/s) — that's why we report the median, not the best run.
• A curiosity: with speculative decoding, temperature 0.6 is faster than 0 (65.0 vs 47.1 tok/s). With sampling, verification accepts more of the drafted tokens.

DeepSeek V4 Flash benchmark on DGX Spark: tokens per second

In daily use (262K profile, about 39% speculation acceptance) DeepSeek V4 Flash generates 44–53 tokens per second in one conversation. In a controlled series on August 19, the median on prose was 63.5–65.0 tok/s. In the long profile (900K-token window, NVFP4 KV cache) it's about 42 tok/s measured by us.

How many speculative tokens: 5, not 7

The vLLM recipe runs DSpark with 7 tokens ahead. On GB10 we tried 2, 3, 5 and 7 — the optimum is 5. At 7, prose acceptance drops to 27.2%: the module guesses too far ahead and most proposals are thrown away. Seven only wins on code, which is more predictable.

Speculative token sweep

tok/s and acceptance, median of 6 · measured by us August 19, 2026 · 2× DGX Spark, apps alongside · ■ prose ■ code

2 tokens

prose

49.9acceptance 75.5%

code

54.3acceptance 87.0%

3 tokens

prose

41.8acceptance 47.9%

code

50.2acceptance 63.8%

5 tokens — our setting

prose

63.5acceptance 56.4%

code

53.3acceptance 47.8%

7 tokens — as in the vLLM recipe

prose

42.8acceptance 27.2%

code

58.7acceptance 48.5%

Speed depends on the content

Same model, same setup, different tasks. The more predictable the text, the more drafted tokens pass verification and the faster the answer. Lists of facts are slowest, because each next fact is hard to guess.

Speed by content type

tok/s, one conversation, DSpark 5 tokens · measured by us August 18, 2026 · 2× DGX Spark, apps alongside

code

56.3acceptance 52.4%

math

54.9acceptance 45.7%

prose

47.3acceptance 37.3%

lists of facts

42.7acceptance 32.7%

Several conversations at once

With several conversations at once, each gets less, but total throughput grows. With six conversations the cluster delivers 153.5 tokens per second in total. That's a single measurement series from August 18, so treat it as indicative.

Total throughput with several conversations

tok/s in total · measured by us August 18, 2026, single series · 2× DGX Spark, apps alongside

2 conversations

127

4 conversations

137.5

6 conversations

153.5

Long context: reading at about 2,300 tokens per second

A number rarely reported, and the most important one for document work: how fast the model reads its input. DeepSeek V4 Flash does it at about 2,300 tok/s, growing linearly with length — a 100K-token document starts its answer after about 43 seconds. The second turn is much faster: the prefix cache hit 98.7% of the time, so an agent loop only processes what's new.

Time to read the input

seconds until the answer starts · measured by us (100K and 250K calculated from the ~2,300 tok/s rate) · 2× DGX Spark

16K tokens

7.9 s

64K tokens

25.7 s

100K tokens

≈43 s

192K tokens

81.8 s

250K tokens

≈1:48

What limits speed on GB10

• Memory bandwidth, not compute. The GPU is 84–95% busy but draws only 30–35 W — it's waiting for memory.
• The 200 Gb/s link isn't the bottleneck. During generation it's 1.66% utilized (3.32 Gb/s). A second cable won't help.
• No throttling. The clock stays at 2,398–2,405 MHz, at 60–64 °C.
• Tuning that did nothing: MTU 9000, speculation lengths other than 5, expert parallel (it doesn't start with the MoE backend we use), changing swappiness. The default recipe was already optimal — that's useful to know too.

Quality: control tests and real documents

First, a control set: 26 tasks in eight categories, checked automatically (code is run against assertions). DeepSeek V4 Flash passed all of them measured by us August 19. It's a safety net that catches a broken setup, not a measure of the model's class.

Control set: 26/26

measured by us August 19, 2026

4/4

reasoning — thinking grows with difficulty: 302 → 1,371 characters

4/4

math

3/3

long context — needle at 8K, 60K and 180K (180K in 27.4 s)

3/3

tools — including not calling a tool when it isn't needed

2/2

format — clean JSON, exactly 5 lines

4/4

assertiveness — rejected false premises

4/4

code — run against assertions

2/2

Polish (our working language)

Assertiveness is our favorite test: the model gets a question built on a false premise and should reject it instead of politely answering. DeepSeek wouldn't accept that Sydney is the capital of Australia, that 17 is even, that water boils at 90 °C, or that dividing by zero is “normal”.

Real documents

Measured by us August 19, 2026. Timing: 92K tokens — about 60 s to read, 15 s thinking, 8 s answer; 120K — about 2 min, 20 s and 60 s.

We checked both AI Act errors against the regulation. The model assumed that the open-source exemption in Article 53(2) exempts providers from all of Article 53 — it only exempts them from points (a) and (b) of paragraph 1. It also said general-purpose AI models must be registered in the EU database, although Articles 49 and 71 apply only to high-risk systems listed in Annex III. Takeaway: DeepSeek V4 Flash is stronger on numbers than on law. For legal work, require citations (“give the article and paragraph number”).

Coding agent

We gave the model file tools and a code sandbox, then three tasks:

Two changes helped: a bigger agent context (from 30,400 to 200,000 tokens) and one rule in the instructions: “if a test fails, assume the bug is in the code, not in the test”. After that, not a single test was bent. kvstore durability: 50/50 writes survived a hard kill of the process, and 600/600 keys with four threads writing at once.

For agent work, DeepSeek also released an official harness, dsh (MIT, developer preview from August 13, 2026): it reads and writes files, runs commands and closes the “run → error → fix” loop on its own. It isolates the agent at the kernel level (we checked by trying: it can't write outside its directory), with about 150 MB of overhead. Note: by default its telemetry sends the session log with the /feedback command — we turn it off on every deployment.

Silent failures: what cost us the most time

None of these ends with a clear error. The model runs, just badly — and it's easy to conclude that it's “weak”.

• The generation_config.json file in the weights forces temperature 1.0 and top_p 1.0. The model could then drift into HTML or wikitext. We set 0.6 and 0.95. It looks like a “dumb model”, but it's only the sampling.
• vLLM returns thinking in the reasoning field, not reasoning_content. An app that reads only the second one shows the model as “not thinking”.
• earlyoom killed vLLM at large contexts — we turned it off.
• Expert parallel doesn't start with the flashinfer_b12x MoE backend.
• A GB10 out of memory freezes instead of killing the process. Ping works, SSH doesn't, only the power button helps — other users report the same. That's why we keep at least about 10 GB free on each unit and set gpu-memory-utilization lower than the recipes do.
• The wrong RDMA device silently costs about 40% of performance. Each unit has several RDMA devices, and the first on the list is usually the wrong one. We map them by the physical port the cable is plugged into.

License: can you use DeepSeek V4 Flash commercially?

Yes. The repository and weights are under the MIT license — one of the most permissive: you may use the model commercially, modify it and offer it to customers, as long as you keep the license notice. For a company that wants to offer the model to customers or partners, that's the simplest possible position — the same as with GLM 5.3 Flash, and unlike Qwen3.8-Flash-Next, whose license requires a separate agreement before offering the model to third parties. The dsh harness is MIT too.

Compared with GLM 5.3 Flash and Qwen3.8-Flash-Next

On the same cluster and with the same method we also measured GLM 5.3 Flash and Qwen3.8-Flash-Next (in September and October; DeepSeek in August, so we compare with care). DeepSeek reads long documents fastest (about 2,300 tok/s vs about 1,240–1,690 for GLM) and is the most thoroughly tested on documents and agent work. Qwen3.8-Flash-Next starts answering fastest (142–161 ms to the first token) and leaves the most free memory (24–26 GB per unit); with six conversations at once both reach about 150 tok/s in total. GLM 5.3 Flash on vLLM is the slowest (24.3 tok/s on prose), although its maker says it's stronger at code and agent work. For offering a model to customers, DeepSeek and GLM have the simplest license — MIT. The full side-by-side comparison, with charts and seven lessons from running them: Best LLM for 2× DGX Spark.

DeepSeek V4 Flash FAQ

Can you run DeepSeek V4 Flash locally?

Yes. We run it on a cluster of two DGX Spark / ASUS GX10 units linked with a QSFP56 200 Gb/s cable, in vLLM with the DSpark speculative decoding module. In daily use it generates 44–53 tokens per second in one conversation, and about 150 tokens per second in total with six conversations at once.

How much memory does DeepSeek V4 Flash need? Does it fit on one DGX Spark?

The DSpark checkpoint we use takes 156 GB on disk and 79.2 GiB of weights per unit when split across two, so it needs two 128 GB units. According to Unsloth, the 3-bit GGUF UD-IQ3_XXS (about 103 GB) fits on a 128 GB machine — in theory a single DGX Spark. We haven't tested that version.

How many tokens per second does DeepSeek V4 Flash get on DGX Spark?

On our 2× DGX Spark cluster: 44–53 tok/s in daily use, a median of 63.5–65.0 tok/s on prose in a controlled series (August 19, 2026), and 153.5 tok/s in total with six conversations at once. It reads long inputs at about 2,300 tok/s, so a 100K-token document starts its answer after about 43 seconds.

Can DeepSeek V4 Flash be used commercially?

Yes. The DeepSeek V4 Flash repository and weights are under the MIT license, which allows commercial use, including offering the model to your customers.

How does DeepSeek V4 Flash handle legal documents?

In our test on the full text of the EU AI Act (about 120K tokens) it made 2 errors in about 25 legal statements: it overstated the scope of the open-source exemption in Article 53 and attributed a registration duty to general-purpose AI models that applies only to high-risk systems. It got every number right. For legal work, require citations with article numbers.

Sources

• DeepSeek, model cards on Hugging Face: DeepSeek-V4-Flash (284B / 13B active, 1M context, MIT, preview), DeepSeek-V4-Flash-0731, DeepSeek-V4-Flash-DSpark and DeepSeek-V4.1-Flash.
• vLLM Recipes, DeepSeek-V4-Flash-0731 — DSpark from vLLM 0.25.0, a 7-token config, two DGX Spark units at tensor parallel 2, deepseek_v4 parsers.
• MiaAI-Lab, DeepSeek V4 Flash with DSpark on 2× DGX Spark (recipe).
• Anemll, dspark-vllm-gx10 — a vLLM image with DSpark support for DGX Spark / GX10.
• NVIDIA Developer Forums, running DeepSeek V4 Flash with DSpark on DGX Spark.
• DeepLearning.AI, DeepSeek's DSpark Gains Velocity (July 10, 2026) — how DSpark works, 60–85% faster than MTP-1.
• Unsloth, DeepSeek-V4-Flash-GGUF and the run guide — UD-IQ3_XXS about 103 GB, at least 110 GB of memory recommended, DSpark in llama.cpp.
• Regulation (EU) 2024/1689 (AI Act) — Articles 49, 53 and 71 and Annex III, against which we checked the model's answers.
• Measurements and tests: our own, ailone's 2× DGX Spark / ASUS GX10 cluster, August 18–19, 2026.

As of October 11, 2026. Engines and images change every week — the numbers apply to the setup described above. Measurements are medians from one series, with apps running alongside.