Overview
- PrismML has launched Bonsai 27B, a 27-billion-parameter AI model that has been compressed to just 3.9 GB, allowing it to operate on an iPhone 17 Pro Max at a speed of 11 tokens per second. This marks a significant achievement as it is the first model of its capability to fit within a smartphone's memory constraints.
- The ternary version of the model maintains 94.6% of the performance of full-precision benchmarks, surpassing traditional "2-bit" Qwen models that are nearly twice as large and struggle with math and coding tasks that require precision below 4 bits.
- According to CNBC, Apple is in preliminary discussions with PrismML regarding the underlying compression technology, with plans for a compressed Gemma model to follow.
AI models typically require significant memory resources. A 27-billion-parameter model is considered medium-sized and usually needs about 54 GB of memory to function effectively at half precision, which exceeds the capacity of most laptops and even some desktops.
This week, PrismML unveiled a model that fits within just 3.9 GB—small enough for use on an iPhone.
The term "parameters" refers to the adjustable components of a model; a higher number of parameters indicates a more complex and capable model.
Bonsai 27B is the first of its class to successfully operate within the memory limits of consumer smartphones, achieving a speed of 11 tokens per second on an iPhone 17 Pro Max. (Tokens are the fundamental units of information processed by AI models.) The ternary variant, weighing in at 5.9 GB, can process around 26 tokens per second on an M5 Pro laptop. Both versions are freely available under the Apache 2.0 license.
The compression technique, based on Caltech's intellectual property, reduces model weight from 16 bits of floating-point precision to a single sign—either +1 or -1 in binary form, or one of three values in ternary form. Each set of 128 weights shares a 16-bit scaling factor, resulting in the binary variant being 1.125 bits per weight, which is 14 times smaller than the original full-precision model. The ternary model extends this by including a zero state for greater expressiveness, arriving at 1.71 bits per weight.
In simpler terms, this means that a ternary AI model relies on three possible settings (negative, zero, or positive) for each internal value, whereas a traditional AI model can utilize around 65,000 settings.
Remarkably, PrismML achieved this level of compression without significantly sacrificing output quality.
What sets this apart from standard "low-bit" models is that all components—including embeddings, attention mechanisms, and the full language model head—are compressed uniformly. Most quantized models maintain certain sensitive layers at full precision, which increases their size to enhance quality. Bonsai avoids this approach altogether.
This is the second significant release in the Bonsai series. In March, PrismML introduced Bonsai 8B, a 1.15 GB model demonstrating that a 1-bit architecture could function effectively at 8 billion parameters without compromising reasoning capabilities. The leap to 27 billion parameters marks a critical threshold where consistent chain-of-thought reasoning, reliable tool usage, and multi-step agentic behavior emerge—areas where smaller models often struggle.
Performance Metrics
In evaluations across 15 benchmarks using NVIDIA H100 GPUs—covering knowledge, mathematics, coding, and tool usage—the Ternary Bonsai 27B achieved an average score of 80.49, representing 94.6% of the full-precision model’s performance. The 1-bit variant scored 76.11.
In general, these models outperform Gemma 4 and Qwen 3.6, especially regarding their potential relative to size.
The performance of these models is impressive considering their minimal resource requirements, pushing the capabilities of small devices (smartphones and lower-end PCs) to new heights. For instance, AIME25 and AIME26, based on the American Invitational Mathematics Examination, yielded scores of 93.7% for Ternary Bonsai 27B compared to 95.3% for the larger Qwen 3.6B. Bonsai scored 86 in coding tasks while Qwen 3.6 scored 88, and Bonsai achieved 77% in general knowledge compared to Qwen 3.6’s 83.
The model also employs a hybrid attention architecture, with around 75% of its layers utilizing linear attention instead of full quadratic attention. This design enables a practical context window of 262K tokens on-device, which would be prohibitively expensive with a standard attention structure on mobile hardware.
Our Testing Experience
We conducted our own tests with Bonsai 27B. Coding requires iterative processes; single-shot prompts do not match the performance of cloud models. However, being local and free makes this distinction less relevant. For our Zombie Type game—a first-person typing horror browser game—two rounds of coding produced accurate collision detection, proper scoring logic, and consistent graphics. The model effectively understands structure early on; the second pass enhances rather than reconstructs the work.
Interestingly, some models (like the skeletons) appeared more detailed than those generated by GPT 5.6 Sol. This does not imply superiority, but rather that in this instance, the model created a more visually appealing skeleton compared to the AI king's choice.
The game can be tested here.
Creative writing presents a different challenge, as the evaluation criteria are more subjective.
Generally, the outcomes are not particularly innovative when prompted with zero-shot inputs.
Nonetheless, Bonsai generates narratives with cohesive internal logic, pacing, and structure—comparable to or better than Claude Haiku or Sonnet under similar prompts, with less effort involved. For a model that operates entirely on local hardware without API fees, that is notable.
The narrative it produced can be found in our Github repository.
Alongside the model, PrismML has also introduced a DSpark speculative decoding layer, a lightweight tool that proposes blocks of candidate tokens, which the primary model verifies in a single forward pass instead of generating tokens one by one. On an H100, this results in a 1.37x throughput increase without any degradation in output quality, as verification retains the exact output distribution. However, it is not yet enabled by default for Apple Silicon, but it represents a significant advantage for GPU serving.
Apple's interest adds a commercial aspect. PrismML CEO Babak Hassibi confirmed to CNBC that discussions are ongoing with Apple regarding the evaluation of the compression technology for potential use on devices.
Hassibi mentioned that a compressed Gemma model is the next project in the pipeline, followed by larger frontier models; the 1-bit Bonsai 27B is currently available for free download under Apache 2.0. For those interested in running models like this locally, check out our guide.