On-device AI
On-device AI is machine learning that runs on the user's own phone or computer rather than on a server, so the data being processed stays on the machine.
On-device AI means the model runs where the data already is: the phone or computer in front of you, not a data center. It became practical for consumer software between 2024 and 2026, when every major platform shipped a system model applications can call. Apple exposes its Foundation Models through Apple Intelligence, Android ships Gemini Nano, and Windows runs models on the NPUs in Copilot+ machines. An application no longer has to bundle a model; it asks the operating system for one.
The trade is capability for locality. A system model is small, so it is good at bounded jobs: pulling structured facts out of text, reading a card or a screenshot, classifying, summarizing short inputs. It is not a frontier reasoning engine, and treating it as one produces bad answers with local latency. Mature designs route instead: the job runs on the device when a small model does it well, and goes to a large hosted model when it does not. The routing decision, not either engine alone, is the architecture.
What running locally buys is concrete. Latency drops to what the silicon can do, marginal cost drops to zero, the feature works offline, and the raw input never has to be transmitted to be understood. That last property is the interesting one for personal data, because a hybrid system can keep the raw material local and move only the structured result, which changes what the product has to be trusted with.
Why it matters
The label is doing heavy marketing work right now, and it often covers only part of a pipeline. The useful evaluation questions are which jobs actually run on the device, what leaves it, and what happens when the local model is unavailable. A product with a clear answer to all three has an architecture; one without has a slogan.