PrismML Unveils Bonsai 27B: A 27B-Class AI Model Running on Smartphones

PrismML has released Bonsai 27B, the first 27B-class AI model capable of running on a smartphone, based on Qwen3.6 27B. This advancement enables persistent, offline AI agents with multimodal capabilities and 262K-token context on devices like the iPhone 17 Pro. For broader context, explore our AI Tools Pricing.

On-Device AI with Bonsai 27B

Bonsai 27B introduces two primary variants, each optimized for different deployment scenarios through advanced quantization techniques. The 1-bit variant, specifically engineered for mobile devices, has a file size of 3.9 GB. This compact size allows it to run on devices such as the iPhone 17 Pro, making advanced AI capabilities available without cloud connectivity. This variant retains approximately 90% of the performance of its full-precision baseline across 15 benchmarks.

For laptop environments, PrismML offers a ternary variant of Bonsai 27B. This version is 5.9 GB and maintains 95% of the full-precision baseline performance across the same 15 benchmarks. Both variants are multimodal, supporting both vision and text inputs, and feature a substantial 262K-token context window. They also incorporate speculative decoding to enhance processing speed.

Performance and Efficiency Metrics

The efficiency of Bonsai 27B is evident in its performance metrics. When tested on an M5 Max, the 1-bit variant achieved a processing speed of 87 tokens per second (tok/s). The ternary variant, also on an M5 Max, reached 58 tok/s. These speeds indicate practical usability for on-device applications.

PrismML highlights the model's intelligence density, a metric that quantifies performance per gigabyte. The 1-bit Bonsai 27B achieves an intelligence density of 0.53 per GB, which PrismML states is over ten times higher than the full-precision baseline. This efficiency is crucial for deploying complex AI models on devices with limited memory.

Quantization and Benchmark Results

The core of Bonsai 27B's on-device capability lies in its extreme quantization. The 1-bit variant operates with an effective 1.125 bits per weight, significantly reducing its memory footprint from approximately 54 GB in its 16-bit full-precision form to 3.9 GB. This reduction is critical for fitting within the memory constraints of modern smartphones.

Despite the aggressive quantization, the model's performance on specific tasks remains robust. Math and coding benchmarks, for instance, show minimal degradation. The 1-bit variant scored 91.7 in math and 81.9 in coding, compared to the full-precision baseline scores of 95.3 and 88.7, respectively. This suggests that critical reasoning capabilities are largely preserved.

Implications for Offline AI Agents

The release of Bonsai 27B under the Apache 2.0 license means it is fully open source, allowing broader access and development. Its ability to run entirely on-device enables the creation of persistent AI agents that function offline. This eliminates the need for continuous cloud access and incurs zero marginal cost per inference step, offering new possibilities for privacy-focused and cost-effective AI applications. Such agents could support multi-step reasoning, tool calling, and various computer-use agentic capabilities directly on personal devices.

Conclusion

PrismML's Bonsai 27B represents a notable advancement in making large language models accessible on consumer hardware. By enabling a 27B-class model to run on smartphones and laptops with significant performance retention, it opens avenues for new offline AI applications and reduces reliance on cloud infrastructure. This development could influence the future of AI news and on-device processing.

Sources

• prism-ml/Bonsai-27B-mlx-1bit
• prism-ml/Bonsai-27B-unpacked · Hugging Face
• prism-ml/Bonsai-27B-gguf
• prism-ml/Ternary-Bonsai-27B-gguf
• https://prismml.com/news/bonsai-27b