Bonsai 2 2B: a tiny vision model, with a few big setup decisions

A 0.52 GB local AI model that can look at an image sounds like an easy win. Here's the catch: 0.52 GB is the language-model file in Prism ML's PQ2_0 packing. To actually ask about a photo, you also need the recommended 0.36 GB vision projector, a compatible runtime, and enough working memory for the model, image processing and context. Small download? Yes. A complete 0.52 GB vision setup? No.

That distinction makes Ternary Bonsai 2 2B a useful little project. The Apache 2.0 GGUF weights are available now. You can learn how local multimodal inference fits together, then decide whether it makes sense for a private photo helper, an offline document sorter or a camera-fed prototype. I haven't run these commands or benchmarked the model myself; this is a guided starting point from Prism ML's model card and demo documentation.

First, know what you're downloading

Bonsai 2 2B is a vision-language model derived from Qwen3.5-2B. Its language side is about 1.88 billion parameters; a separate vision tower adds about 0.33 billion. Many of the language weights use three values—minus one, zero and plus one—with a scale per group. That's why the files can be small. It doesn't mean every tensor is ternary or that the whole running process fits in the file size.

There are three GGUF language packings of the same weights. PTQ1_0 is 0.43 GB and the smallest on disk. PQ2_0 is 0.52 GB and Prism ML's default choice for its fork. Q2_0 is 0.55 GB and the route to upstream llama.cpp. Add the Q8_0 vision projector, 0.36 GB, for image input. So the PQ2_0 pair is about 0.88 GB of downloads before runtime overhead; the Q2_0 pair is about 0.91 GB. Those sums are storage estimates, not RAM or VRAM requirements.

The low-friction route: use Prism ML's demo

For a first pass, follow the linked Bonsai demo repository instead of guessing which binary understands which packing. Its setup scripts select a model, download matching runtime binaries and provide launchers. The demo defaults to the much larger 27B model, so choose 2B explicitly before running setup. On macOS or Linux, the documented flow is: git clone https://github.com/PrismML-Eng/Bonsai-demo.git, cd Bonsai-demo, export BONSAI_MODEL=2B, then ./setup.sh. Start the local server with ./scripts/start_llama_server.sh. On Windows PowerShell, set $env:BONSAI_MODEL = "2B" before using the repository's setup.ps1 and Windows launcher. Check the repository's current platform instructions before you run anything; setup may install dependencies and download more than the two model files.

For a leaner experiment, Prism ML's model card shows the direct GGUF path. Download the PQ2_0 language file and Q8_0 mmproj from the model page, use a current binary from Prism ML's llama.cpp fork, and try text first: ./bin/llama-cli -m Ternary-Bonsai-2-2B-PQ2_0.gguf -ngl 99 -fa on -p "Explain this in three sentences." Then add vision with ./bin/llama-mtmd-cli -m Ternary-Bonsai-2-2B-PQ2_0.gguf --mmproj Ternary-Bonsai-2-2B-mmproj-Q8_0.gguf -ngl 99 --image photo.jpg -p "Describe this image." Change photo.jpg to a local test image. -ngl 99 asks for GPU offload; Prism ML also documents CPU and other backends, so use its platform guidance if your machine differs.

If you want stock llama.cpp, change the file too

Upstream llama.cpp supports the Q2_0 packing from release b11556 onward, according to Prism ML. It does not support PTQ1_0 or PQ2_0 yet. Download Ternary-Bonsai-2-2B-Q2_0.gguf plus the same Q8_0 mmproj, and use an upstream build at least that recent. Older builds can produce wrong answers without warning because the model needs a matching activation transform. That's a nastier failure than a clean “unsupported file” error, so check the version before evaluating output.

Don't pick the 0.43 GB PTQ1_0 file just because it wins the size contest. Pick the runtime first, then its supported packing. And don't use the larger BF16 vision projector as an automatic “quality upgrade”: the model card recommends Q8_0 and warns that BF16 can crash its current fork's server under concurrent image requests.

A useful first test, not a victory lap

Start with ten ordinary images you can share locally: a simple object, a cluttered desk, a screenshot, a receipt, small printed text and a few deliberately ambiguous scenes. Ask the same question each time. Note whether it gets the important detail right, how long the answer takes, and whether memory use stays comfortable when images and longer prompts arrive. For small or dense text, Prism ML suggests trying --image-min-tokens 1024; that is a trade-off to test, not a magic accuracy switch. Keep sensitive documents off any networked test service unless you understand where requests go.

The advertised 262K-token context is an architectural limit inherited from the base model, not a promise that a tiny machine can run a 262K-token session cheaply. Start with a modest context and measure actual RAM or VRAM, latency and accuracy on your hardware. If the result is good enough, you will have learned a portable skill: matching model weights, vision projectors and runtimes, then budgeting the whole application rather than celebrating one small file. That is more useful than another impressive download figure.