What is an NPU?

An NPU (Neural Processing Unit) is a processor specialized in the math of neural networks, sitting next to the CPU and the GPU inside the same chip of a laptop or a phone. It is not one vendor's product but a category of accelerator: Qualcomm, Intel and AMD each sell their own version under a different commercial name. Its job is to run inference on far less energy than the CPU or the GPU would need for the same work, which is what makes it matter on a laptop or a phone more than raw throughput does. Microsoft's developer guide defines a Copilot+ PC as a class of Windows 11 machines with an NPU able to perform more than 40 trillion operations per second (40+ TOPS), and adds that many Windows AI features require exactly that threshold. What no spec sheet spells out is the condition attached: an NPU only works for software written specifically to use it.

See also: AI accelerator lock-in · The cost of moving an inference workload to another accelerator: model weights are portable, optimizations are not.

How it differs from CPU and GPU

The CPU is a generalist and runs anything well, one thing at a time. The GPU was born for parallel floating point math and remains the default for training and for large models. The NPU is narrower than both: it does few operations, the deep learning ones, at a performance per watt the other two cannot match. That specialization carries a precise technical price. Models are trained in wide formats such as FP32, while many NPU devices, Microsoft writes, only support integer math in lower bit formats such as INT8: before it can run on the NPU a model has to be quantized, that is converted, with the accuracy check that follows. Then there is a behavior to know before signing a purchase order. Windows ML queries the machine, picks the most performant execution provider available on its own and, if the preferred one is missing or fails, falls back to the GPU or the CPU. It is deliberate behavior, and a convenience for whoever is developing; for whoever bought the machine it means the application keeps working while the workload may never touch the NPU.

An enterprise example

A design firm with forty workstations renews its fleet and finds two configurations of the same model, a few hundred euros apart, one of them badged Copilot+ PC, that is with the 40+ TOPS NPU Microsoft requires for that program. The right question is not what the TOPS are worth, but which applications in use exploit them: the AI features built into Windows do by construction, video calls and transcription depend on how the vendor wrote them, the industry specific line of business tool almost never does, because nobody recompiled it for an NPU. Across forty machines the difference is a real budget line, and it gets decided with a trial: two pilot workstations, a week of real work, any planned local language model put under load, and NPU utilization read in the Windows Task Manager, where Microsoft shows it next to CPU and GPU.

Why it matters for decision makers

It matters because an NPU's value lives in the software the company uses every day, not in the spec sheet. Buyers should ask their software vendors, in writing, whether and when their products will use the accelerator, instead of inferring it from the presence of the chip. The reverse holds too: where the software exists, the NPU moves work from the cloud to the desk, which is edge AI applied to the office, with data that never leaves and no per call cost. Finally, the choice between silicon vendors is not neutral over time, and accelerator lock-in is what tells you the cost of changing your mind. The Microsoft guide to NPU devices states both the threshold and the fallback to GPU and CPU.