PrismML has launched Bonsai 27B, a compact AI model that operates on smartphones, achieving impressive performance metrics.
Overview
PrismML has introduced Bonsai 27B, a groundbreaking AI model with 27 billion parameters compressed to just 3.9 GB, enabling its operation on devices like the iPhone 17 Pro Max at a processing speed of 11 tokens per second. This achievement positions Bonsai as the first AI model of its kind to function within a smartphone's memory limitations.
The ternary version of Bonsai 27B retains 94.6% of the performance levels of traditional full-precision models, outperforming larger "2-bit" Qwen models that struggle with precision tasks. The model's compact size is particularly notable as typical AI models of this scale require around 54 GB of memory, which is beyond the capacity of most consumer hardware.
PrismML's innovative compression technique, derived from Caltech's intellectual property, reduces the model's weight significantly, allowing it to operate efficiently without compromising on output quality. The ternary model uses three possible states for each internal value, enhancing expressiveness while keeping the model size manageable.
Performance Metrics
In tests across 15 benchmarks with NVIDIA H100 GPUs, the Ternary Bonsai 27B achieved an average score of 80.49, demonstrating its competitive edge over models like Gemma 4 and Qwen 3.6, especially regarding size-to-performance ratio. Notably, Bonsai scored 86 in coding tasks and 77% in general knowledge assessments, indicating its robust capabilities despite being housed on mobile devices.
Testing Insights
Initial tests of Bonsai 27B indicate that while it may not match the performance of cloud-based models in all scenarios, its local operation and zero cost make it an attractive option. In a coding exercise for a game, Bonsai effectively handled the logic and structure, showing promise in practical applications. However, creative writing tasks revealed the model's limitations in generating highly innovative narratives.
Additionally, PrismML has launched a new speculative decoding layer called DSpark, enhancing throughput without sacrificing quality. Discussions with Apple about integrating this technology into their devices are underway, with plans for future model developments on the horizon.
