We use cookies for ads and analytics to support our work. You can accept or reject these non-essential cookies. Cookie policy.
Phones now run LLMs and image models locally on NPUs. How Apple Intelligence, Galaxy AI, and Pixel do it, plus the privacy, speed, and tradeoffs.
Ask your phone to summarize a long webpage, translate a call in real time, or erase a stranger from a photo, and there's a good chance the AI never reaches a data center. Increasingly, the model runs on a dedicated chip a few millimeters from the camera.
Modern phones ship with a Neural Processing Unit (NPU): silicon built to run the matrix math behind neural networks far more efficiently than a CPU or GPU. The big three are Apple's Neural Engine, Qualcomm's Hexagon NPU in the Snapdragon 8 Elite, and the custom TPU inside Google's Tensor.
To fit a useful model into a few gigabytes of RAM, vendors shrink it through quantization — storing each weight with fewer bits. Apple's on-device foundation model holds roughly 3 billion parameters compressed to about 2 bits per weight: small enough to load on a phone, yet capable enough for everyday text tasks. The same NPUs also run scaled-down diffusion models, which is how generative photo edits and stickers can render entirely on the handset without a server.
All three major platforms now run a small language model locally and fall back to the cloud only for heavier jobs.
| Platform | Silicon | Local model | Example on-device features |
|---|---|---|---|
| Apple Intelligence | A-series + Neural Engine | ~3B foundation model | Writing Tools, notification summaries, Genmoji |
| Galaxy AI | Snapdragon 8 Elite | Gemini Nano + Samsung models | Live Translate, generative photo edits |
| Pixel | Tensor G5 | Gemini Nano v3 | Magic Cue, Voice Translate, on-device Scam Detection |
Google says Gemini Nano runs roughly 2.6x faster on the Tensor G5 than the prior generation, which is why fully on-device, real-time call translation became practical on the Pixel 10.
On-device AI is a tradeoff, not a free lunch.
The industry's answer is hybrid routing: simple requests stay local, hard ones go to a privacy-hardened server. Apple's Private Cloud Compute and Samsung's cloud-based Galaxy AI features both work this way, and Apple lets you restrict processing to the device alone.
The direction is clear: the phone is becoming the default place to run AI, with the cloud reserved for the few tasks a pocket-sized model genuinely can't handle.
An NPU (Neural Processing Unit) is a chip block designed specifically to run neural-network math efficiently. It lets a phone run AI models faster and with far less battery drain than the CPU or GPU could.
Yes. Features that run on the local model, such as text summaries, live translation, or photo cleanup, keep working in airplane mode or with no signal. Tasks routed to the cloud will not.
Recent flagships from all three ecosystems do: iPhones with Apple Intelligence, Samsung Galaxy phones with Galaxy AI on Snapdragon 8 Elite, and Pixel devices running Gemini Nano on Tensor. Most require a current chip and 8 to 12GB of RAM.
Generally yes, because data processed locally never leaves your phone, so it cannot be intercepted or logged on a server. Just note that some advanced features still send data to the cloud when the local model is not powerful enough.
Short tasks are cheap, but long or repeated AI generation warms the phone and uses more power than a quick network call. The NPU is built to minimize this, but heavy use still has a measurable cost.
Phone models are small, around 3 billion parameters, so they cannot match the reasoning and knowledge of cloud models with hundreds of billions of parameters. Vendors use hybrid routing to send only the hardest requests to a server.
Share quick feedback - it's anonymous and separate from comments.
Comments
No comments yet - be the first to share your thoughts.
Leave a comment