Bonsai 2 27B on a regular PC — our hardware test and a 10-minute setup guide

Short answer: which hardware is enough

You need an NVIDIA graphics card with at least 8 GB of memory; 16 GB is the sweet spot. On a desktop with an RTX 5060 Ti 16 GB, Bonsai 2 27B writes an email in 5–7 seconds and summarizes 10 pages in about 30–40 seconds. Without a dedicated GPU it still starts, but every answer takes minutes. A card with 6 GB can't hold the model at all.

What Bonsai 2 27B is

Ternary Bonsai 2 27B is a model from PrismML, released in September 2026 under the Apache 2.0 license. It's a compressed version of Qwen3.8-27B: every weight is stored as −1, 0 or +1 (“ternary”), with one FP16 scale per group of 128 weights, in a basis rotated by a Hadamard transform. Instead of 16 bits per weight that takes about 1.75–2.13 bits, so the 53.8 GB original shrinks to:

• PTQ1_0 — 5.95 GB (about 1.75 bits per weight);
• PQ2_0 — 7.21 GB (about 2.13 bits per weight) — the same weights, packed differently;
• optional vision tower mmproj-Q8_0 — 0.63 GB, needed only for images.

The model keeps what the original could do: a 262,144-token context window, text and image input, thinking (reasoning) modes and tool calling. According to PrismML's whitepaper it averages 83.9 vs. 85.4 for the full-precision Qwen3.8-27B on a 20-benchmark suite — 98.2% of the original (math 99.5%, coding 99.3%, knowledge and reasoning 96.9%, vision 96.3%). The weak spot is long agentic work: on SWE-bench Verified it scores 60.8 vs. 80.6, and on Terminal-Bench 2.1 52.8 vs. 69.7 — roughly three quarters of the original. These are PrismML's numbers.

Because of the special weight format, standard llama.cpp can't run it: it rejects the files or, in one older format, loads them and produces garbage. PrismML ships its own llama.cpp fork (CUDA, Metal, CPU) and an MLX build for Apple Silicon. Ollama, LM Studio and vLLM don't support it as of October 10, 2026.

Bonsai 27B vs. Bonsai 2 27B

Searches for “bonsai 27b” mix up two generations:

PrismML also released a ternary version of the first generation. Everything on this page is about Ternary Bonsai 2 27B.

Our test: quality

We tested Bonsai 2 27B on September 25, 2026 with eight office tasks in Polish — our working language, and a harder test than English: a legal question, an email, a 10-page summary, a 30-page contract analysis, an invoice converted to JSON, a tool call, a coding task and an invoice scan (OCR).

• 24 of 24 automatic checks passed: every fact from the terms and the contract with the right section numbers, invoice to JSON without a single error, a correct tool call, code passing 16 of 16 hidden tests, OCR of a scan with Polish characters, and a fact found in a document of 116,000 tokens (~350 pages).
• Blind review by GLM 5.3 Flash — Zhipu AI's 320-billion-parameter model that we use every day on our two-node ASUS GX10 cluster. It saw both answers in random order and scored each from 1 to 10. GLM itself also passed 8 of 8 automatic checks.

Caveats: the judge is a language model too (in one question it cited the wrong article of the code itself), we ran one sample per task, and the documents were synthetic. Treat the result as a solid indication, not a benchmark.

Our test: which hardware is fast enough

We measured Bonsai 2 27B on an ASUS GX10 (NVIDIA GB10). Speeds for consumer GPUs are estimated: scaled from the GX10 measurements and from emulating smaller cards' memory limits, with an accuracy of about ±10–15%. All numbers are for one user, with thinking switched off.

Tasks × hardware: time to a complete answer

Green = comfortable, amber = you'll wait, red = impractical. Estimated, except where noted.

What we measured on the GB10

• Writing speed drops with context length: 35.0 → 25.3 tokens/s at 32K (PTQ1_0); PQ2_0 does 17.2 at 64K and 13.2 at 116K.
• Reading the prompt: about 1,000 tokens/s.
• A compressed KV cache (q8_0 / q4_0) doesn't slow it down — with long context it's even faster, because there are fewer bytes to read.
• GPU memory with a 32K window: PTQ1_0 7.58 GiB, PQ2_0 8.69 GiB (f16 KV cache); with a q4_0 KV cache 6.22 / 7.34 GiB. KV cache per token: f16 64 KiB, q8_0 34 KiB, q4_0 18 KiB.
• 1 / 2 / 4 / 8 conversations at once on the GX10: 30 / 50 / 82 / 107 tokens/s in total (PQ2_0).
• Vulkan instead of CUDA: PQ2_0 runs at 19 tokens/s (63% of CUDA); PTQ1_0 practically doesn't work (1.9 tokens/s).
• CPU only (10 ARM cores): 1.6–3.0 tokens/s.
• 6 GB card: with 8 of 64 layers moved to the CPU, writing drops from 35 to 8 tokens/s.
• Writing speed is limited by memory bandwidth, not compute: an RTX 5060 / 5060 Ti (448 GB/s) is faster than a GX10 (273 GB/s) for a single user. The GX10 wins with a team and with big models.
• Independent confirmation: users on Hugging Face report about 37–40 tokens/s on an RTX 5060 Ti 16 GB and 34 tokens/s on an RTX 3060 12 GB.

What we didn't measure: consumer GPU speeds are estimated, not measured on those cards. We haven't tested Windows, laptops on battery, integrated GPUs (Intel/AMD) or long agentic tasks, and we ran one sample per task.

Which PC to buy or upgrade

• What matters is an NVIDIA GPU with 8 GB or more, not the price of the computer. The sweet spot is an RTX 5060 Ti 16 GB: it fits the larger PQ2_0 packing, a 64K context window (up to about 123K at most) and images without compromises.
• An RTX 3060 12 GB writes slower (28–35 tokens/s, estimated) at a similar price — the 5060 Ti is the better buy.
• Upgrading an office PC with an RTX 5060 Ti 16 GB gives the same results as a new computer. You need a tower case, a free PCIe x16 slot and a power supply of about 450 W — small form factor and mini PCs won't take the card.
• Above that you pay for speed, not capability: an RTX 5070 Ti 16 GB does about 70–87 tokens/s, a laptop RTX 5070 Ti 12 GB about 45–65 (estimated).
• Prices: NVIDIA's launch prices (April 15, 2025) were $429 for the RTX 5060 Ti 16 GB, $379 for the 8 GB version and $299 for the RTX 5060; laptops with an RTX 5060 started at $1,099. Store prices vary. For reference, in Poland in September 2026 a complete PC with an RTX 5060 Ti 16 GB cost about PLN 4,000–5,500 and the card alone PLN 1,899–1,999 (Polish retail).
• Don't pilot on weak hardware. On an office laptop an email takes over two minutes, and people conclude that “AI is slow”.

How to run Bonsai 2 27B — setup guide

Before you start: you need an NVIDIA GPU with at least 8 GB of memory (16 GB recommended), a current NVIDIA driver and about 10 GB of free disk space. Bonsai 2 does not run in Ollama, LM Studio, vLLM or standard llama.cpp. You need PrismML's own build of llama.cpp.

llama-server -m Ternary-Bonsai-2-27B-PQ2_0.gguf -ngl 99 -fa on -c 65536 -np 1 --cache-ram 24576 --port 8080

llama-server -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 -fa on -c 24576 -ctk q8_0 -ctv q8_0 -np 1 --cache-ram 24576 -fitt 256 --port 8080

Thinking modes: when to switch them off

Bonsai 2 thinks before it answers, and by default at the highest level (xhigh) — often thousands of hidden tokens. In our test that was a trap for office work:

• with thinking off (none), Bonsai passed 100% of the checks on emails, summaries, invoices and scans and answered 3–8× faster;
• with medium it wrote a summary using 5,842 thinking tokens (3–4 minutes) for no gain, and in one email it invented details — a non-existent discount code and the wrong weekday;
• xhigh used 7,177 thinking tokens on a short coding task and 8,288 on a contract.

Our defaults: none for emails, summaries, invoices, OCR and chat; medium for contract analysis, code and agents; never xhigh on consumer hardware. high isn't supported (it returns HTTP 500), and low behaves almost like xhigh.

Set it per request through the OpenAI-compatible API. none switches thinking off as long as the server runs with the default --reasoning auto:

{"model": "bonsai", "messages": [...], "reasoning_effort": "none"}

For the built-in chat you can set it on the server instead: --reasoning off switches thinking off for every conversation, and --reasoning-effort medium makes medium the default level. Both are documented flags of PrismML's llama-server.

Recommended sampling (PrismML): without thinking — temperature 0.7, top_p 0.8, top_k 20, presence_penalty 1.5; with thinking — temperature 1.0, top_p 0.95, top_k 20, min_p 0.05.

Known issues and gotchas

• Exactly one system message, at the start of the conversation — a second one returns HTTP 500. Check this when connecting Open WebUI or LibreChat.
• The server prints its banner and exits → you're on the CUDA 13.3 build; use 12.4 (Windows) or 12.8 (Linux). On some CPUs the Windows CUDA build doesn't start at all; PrismML suggests the CPU-only build meanwhile (slow).
• Empty or cut-off answers: thinking counts against the output limit. PrismML recommends -n 16384 or more together with -c 65536.
• KV cache types: f16, q8_0 and q4_0 only — q5_0 is several times slower. If you build from source with CUDA, a quantized KV cache with flash attention may need -DGGML_CUDA_FA_ALL_QUANTS=ON.
• Don't send thinking back in agent loops: when earlier reasoning and tool calls are re-sent, the context grows fast and the prompt cache stops working.
• Tool calls are occasionally malformed; empty tool arguments return an HTTP error — send "{}" for calls without arguments.
• RTX 5090: prebuilt Linux binaries decode erratically — PrismML recommends building from source with -DCMAKE_CUDA_ARCHITECTURES=120a-real.
• AMD GPUs and Intel GPUs: not recommended for now — PrismML lists open ROCm/HIP, SYCL and Vulkan issues. With Vulkan, the current PQ2_0 binaries can silently fall back to the CPU (under 2 tokens/s).
• Easier route for Windows: PrismML's Bonsai-demo has a setup script, but it downloads only PQ2_0 and sizes the context from system RAM, not GPU memory — on 8–12 GB cards set BONSAI_CTX lower (or BONSAI_KV4=1).

Bonsai 2 27B FAQ

Can Bonsai 2 27B run without a GPU?

Yes, but slowly: we measured 1.6–3.0 tokens per second on 10 ARM CPU cores and estimate 2–4 tokens per second on typical office computers, which means minutes per answer. For daily work you need an NVIDIA GPU with at least 8 GB of memory.

Does Bonsai 2 27B work in Ollama or LM Studio?

No, not as of October 10, 2026. Its ternary formats need PrismML's own build of llama.cpp; standard llama.cpp either rejects the files or produces garbage, and Ollama, LM Studio and vLLM don't support them.

Which Bonsai 2 27B file should I download?

PTQ1_0 (5.9 GB) for 8 GB GPUs, PQ2_0 (7.2 GB) for 12–16 GB GPUs and Macs. Both contain the same weights in two packings. Add mmproj-Q8_0 (0.63 GB) only if you need image input.

Is Bonsai 2 27B as good as Qwen3.8-27B?

Almost, by PrismML's numbers: 98.2% of the original on a 20-benchmark average. In our own blind test it reached about 88% of GLM 5.3 Flash, a model 12 times larger. The gap is biggest in long agentic tasks, where PrismML reports roughly three quarters of the original's score.

What's the difference between Bonsai 27B and Bonsai 2 27B?

Bonsai 27B (July 2026) is a 1-bit version of Qwen3.6-27B: about 3.9 GB and 89.5% of the original's quality. Bonsai 2 27B (September 2026) is a ternary version of Qwen3.8-27B: 5.9–7.2 GB and 98.2%, according to PrismML.

Is Bonsai 2 27B free for commercial use?

Yes. PrismML releases it under the Apache 2.0 license, which allows commercial use.

Sources

• PrismML, Ternary-Bonsai-2-27B-gguf model card — files and sizes, ternary format with Hadamard rotation, 262K context, image input, thinking modes, recommended sampling, license Apache 2.0, standard llama.cpp not supported.
• PrismML, KNOWN_ISSUES (last checked by PrismML September 23, 2026) — system messages, CUDA 13.3, KV cache types, tool calls, RTX 5090, AMD/Intel, Bonsai-demo scripts.
• PrismML, Ternary Bonsai 2 27B whitepaper (September 2026) — 20-benchmark average 83.9 vs. 85.4 (98.2%), category results, SWE-bench Verified and Terminal-Bench 2.1; Bonsai-demo — setup scripts.
• PrismML, Bonsai-27B-gguf (1-bit) — Qwen3.6-27B base, ~3.9 GB, 89.5% of FP16, July 2026.
• PrismML, llama.cpp fork releases and server documentation — file names, --reasoning, --reasoning-effort, --cache-ram, -fitt, reasoning_effort: "none". Checked October 10, 2026.
• Hugging Face, discussion #4 — user reports: ~37–40 tokens/s on RTX 5060 Ti 16 GB, 34 tokens/s on RTX 3060 12 GB.
• NVIDIA, NVIDIA Blackwell GeForce RTX Arrives for Every Gamer, Starting at $299 (April 15, 2025) — RTX 5060 Ti and RTX 5060 launch prices.
• ailone — our own test of Bonsai 2 27B, September 25, 2026: quality on eight tasks, measurements on ASUS GX10, estimates for other hardware.