Models we run and test locally
Open models running on our own hardware — a two-node DGX Spark / ASUS GX10 cluster and regular PCs with NVIDIA graphics cards. Every page says what we measured ourselves and what comes from the model's maker.
Compare the three cluster models side by side: DeepSeek vs GLM vs Qwen →
Qwen3.8-Flash-Next
A preview of the Qwen4 architecture: 125B MoE with only 6B active and a 51B n-gram table. The fastest on our cluster: 150.7 tok/s total with 6 conversations, on stock vLLM. Check the license before offering it to customers.
GLM 5.3 Flash
Zhipu AI's multimodal 320B MoE with a 1M-token window, on two engines: vLLM with NVFP4 (24–32 tok/s, all MIT) and TensorFold with EXL3 (36–71 tok/s, non-commercial). GGUF options and silent failures.
DeepSeek V4 Flash
DeepSeek's 284B MoE with DSpark speculative decoding, run on our two-node cluster: 44–53 tok/s in daily use, 153.5 tok/s in total with six conversations, about 2,300 tok/s reading long documents.
Bonsai 2 27B
A 27B model in a 6–7 GB file. Our test on eight hardware classes, real speeds and quality, and a setup guide for Windows, Linux and Mac.
Decision model benchmark
10 local decision models on 2,198 Polish documents: pplx-decider routes 87.2% of emails automatically, Clef 27B is the most accurate, and prompt injection is everyone's weak spot.
AI decision models
Models that pick an answer instead of writing one: Jev, Clef, Strands Decider, pplx-decider, Nimble and Kev, and which of them run locally.