In short: what is an NPU? A neural processing unit is a chip section built to run AI models efficiently at low power. Its job is not to be fast in absolute terms — a discrete GPU is far faster — but to run small models continuously without draining a battery. That is why NPUs appear in laptops and why the TOPS figure on the box tells you less than it appears to.
Ask what is an NPU and you mostly get marketing. NPUs arrived with a wave of it and very little explanation of what they actually change. The honest answer is narrow but real: they make certain always-on features practical on battery power, and they are largely irrelevant to the heavy AI work most people imagine.
What it is and what it does
A modern laptop processor contains a CPU for general work, a GPU for graphics and parallel maths, and increasingly an NPU for the specific pattern of arithmetic that neural networks perform. Asking what is an NPU is really asking why a third unit is worth the silicon.
The answer is efficiency per watt. An NPU performs the multiply-accumulate operations models rely on using far less power than a CPU or GPU doing the same work. It is not that the NPU is fast; it is that it is cheap to run continuously.
TOPS: the number to read carefully
NPU performance is quoted in TOPS — trillions of operations per second. AMD, for example, publishes its Ryzen AI 400 series at up to 60 NPU TOPS, which we examined in our Ryzen AI 400 explainer.
Three caveats matter when comparing these figures. First, “up to” is doing work — peak throughput under favorable conditions. Second, TOPS figures are usually quoted at reduced numeric precision, and different vendors may not quote the same precision. Third, and most importantly, a higher TOPS number only helps if software actually dispatches work to the NPU, and much software still does not.
Treat TOPS as a rough capability class, not a performance prediction.
What is an NPU compared with a CPU and GPU
| Unit | Best at | Power |
|---|---|---|
| NPU | Small models running constantly — background effects, transcription, wake-word detection | Very low |
| Integrated GPU | Medium models, bursts of work | Moderate |
| Discrete GPU | Large models, training, fast local inference | High |
| CPU | Fallback for anything, and orchestration | Moderate to high |
The practical consequence: if you want to run a large language model locally at speed, the specification that matters is graphics memory, not NPU TOPS. See our local versus cloud AI comparison.
What NPUs are genuinely used for today
- Video call processing — background blur, framing, noise suppression, running for an hour without flattening the battery.
- Live transcription and captions.
- Image and photo enhancement in system apps.
- Wake-word and presence detection, which must run constantly by definition.
- Small on-device assistants handling short, local tasks.
What they are not used for: running large models at useful speed, training anything, or accelerating most third-party applications, which have not been written to target them.
Should it affect your purchase?
Modestly, and less than the marketing implies. Any current mid-range laptop processor includes an NPU, so it is rarely a differentiator between candidates. The specifications that will affect your experience more are memory capacity, sustained cooling and screen quality — see our laptop buying guide.
Where an NPU does matter: long video-call days on battery, and any workflow relying on continuous local processing. Where it does not: gaming, and running large models locally.
Common mistakes
Comparing TOPS across vendors as if identical. Precision and conditions differ.
Expecting an NPU to run large local models. That is a graphics-memory question.
Paying a premium for a higher TOPS figure. Software support decides whether it is used at all.
Assuming existing apps benefit automatically. They must be written for it.
Treating an AI label as a specification. Read the actual figures, then read what uses them.
Verdict
So what is an NPU worth to you? It is a genuine efficiency advance for small, continuous AI work on battery, and close to irrelevant for heavy AI workloads. Buy a laptop on memory, cooling, screen and keyboard; treat the NPU as a welcome inclusion rather than a reason to choose, and read TOPS figures as a capability class rather than a promise.
What we would need to test to say more
Comparing NPUs would require running identical models across devices with power draw measured at the wall and at the battery, confirming which unit executed the work, and testing with software known to dispatch to the NPU. We have not done that. The TOPS figure quoted is a manufacturer value.
Sources and methodology
This article explains documented technique and quotes manufacturer or vendor specifications where stated, linked below. RankBoast has not benchmarked the hardware or models discussed and publishes no performance figures of its own. Research and drafting were AI-assisted. Errors are handled under our corrections policy.
Source links
Join the discussion
Add useful context, ask a focused question or share relevant experience. Comments are moderated to protect readers from spam and promotional links.
Leave a thoughtful comment