A 27B model fits on an iPhone for the first time — 11 tok/s is a demo and Apple is only evaluating it
TL;DR
Bonsai 27B demoed at about 11 tok/s on an iPhone 17 Pro; Apple is only in early evaluation talks.
PrismML said it put a complete 27B-class AI model on a phone for the first time, with a Bonsai 27B demo showing about 11 tokens per second on an iPhone 17 Pro. That is a company demonstration, not an Apple adoption announcement or a production test across millions of phones.
Bonsai is based on Qwen3.6 27B. Its 1-bit build compresses an approximately 54 GB FP16 model to 3.9 GB at 1.125 effective bits per weight; the ternary build is 5.9 GB. PrismML said the 1-bit version retains 90% of the baseline across 15 benchmarks and supports a 262K context.
CEO Babak Hassibi told CNBC that Apple and other companies are measuring speed, energy use, and device performance. He said the Apple talks remain very early. Apple did not comment. Hassibi said all 27 billion parameters can run on an iPhone 15 or newer; the 11 tok/s demo used an iPhone 17 Pro. Those are not the same hardware validation.
PrismML said the 1-bit build scored 76.1 overall, below the baseline's 85.0. The model also runs on NVIDIA GPUs through CUDA, but third-party phone battery, multitasking, and reliability data are absent. Apple has announced no contract, integrated product, or launch date.
via 9to5Mac / PrismML / CNBC
Bonsai is based on Qwen3.6 27B. Its 1-bit build compresses an approximately 54 GB FP16 model to 3.9 GB at 1.125 effective bits per weight; the ternary build is 5.9 GB. PrismML said the 1-bit version retains 90% of the baseline across 15 benchmarks and supports a 262K context.
CEO Babak Hassibi told CNBC that Apple and other companies are measuring speed, energy use, and device performance. He said the Apple talks remain very early. Apple did not comment. Hassibi said all 27 billion parameters can run on an iPhone 15 or newer; the 11 tok/s demo used an iPhone 17 Pro. Those are not the same hardware validation.
PrismML said the 1-bit build scored 76.1 overall, below the baseline's 85.0. The model also runs on NVIDIA GPUs through CUDA, but third-party phone battery, multitasking, and reliability data are absent. Apple has announced no contract, integrated product, or launch date.
via 9to5Mac / PrismML / CNBC
