PrismML has released Bonsai 27B, an open-source multimodal AI model compressed down to 3.9GB that can run locally on an iPhone 17 Pro. Based on Qwen3.6 27B, the model utilizes 1-bit and ternary quantization to retain 90% to 95% of its original baseline performance across standard benchmarks. Developers can now execute complex reasoning, tool calling, and vision tasks directly on-device without cloud API latency or network costs.
Low-Bit Architecture and Speed Benchmarks
PrismML developed the architecture to compress the 27-billion parameter network—which standard 16-bit precision scales to roughly 54GB—without employing higher-precision fallback layers across its attention, MLP, or language model head components. The release features two distinct operating points: a 3.9GB 1-bit variant using binary weights at 1.125 effective bits per weight, and a 5.9GB ternary variant using ternary weights at 1.71 effective bits per weight. Both builds feature a compact 4-bit vision tower, a 262K-token context window, and native support for speculative decoding.
In raw execution speed, the 1-bit build achieves up to 163 tokens per second on an NVIDIA GeForce RTX 5090 GPU and 87 tokens per second on Apple's M5 Max processor. Native framework support is provided out of the box through Apple MLX for iOS and macOS, alongside custom CUDA kernels for NVIDIA platforms.
Evaluating On-Device Model Accuracy
Across a 15-benchmark evaluation suite tested in thinking mode, Ternary Bonsai 27B achieved an overall score of 80.5 and 1-bit Bonsai 27B reached 76.1, compared to the full-precision Qwen3.6 27B baseline score of 85.0. The 1-bit variant delivers an intelligence density score of 0.53 per GB, which is more than 10x higher than the full-precision baseline model.
PrismML's published benchmark performance breaks down as follows across key task areas:
- Math (GSM8K, MATH-500, AIME): Baseline 95.3 | Ternary 93.4 | 1-bit 91.7
- Coding (HumanEval+, MBPP+, LiveCodeBench): Baseline 88.7 | Ternary 86.0 | 1-bit 81.9
- Agentic & Tool Calling (BFCL v3, TauBench): Baseline 80.0 | Ternary 74.0 | 1-bit 66.0
- Instruction Following (IFEval, IFBench): Baseline 78.4 | Ternary 71.8 | 1-bit 65.8
- Vision Tasks (MMMU Pro, OCRBench): Baseline 72.6 | Ternary 65.2 | 1-bit 59.6
Practical Trade-Offs and Deployment Impact
While mathematical reasoning and coding capabilities experience minimal degradation, heavy quantization reveals clear trade-offs in agentic tool integration and visual recognition. Vision benchmarks show a 13 percentage point drop from the 72.6 baseline down to 59.6 on the 1-bit footprint, demonstrating that visual perception suffers under aggressive weight compression. Nonetheless, for software engineers building offline applications, the ability to run multi-step tool loops locally without incurring remote token costs addresses long-standing privacy and operational hurdles.