AI is changing technology everywhere you look. AI tools and functionality are being incorporated into apps and platforms of all types and sizes, with Microsoft arguably leading the charge. We are even getting a new key on our keyboards – the AI key. Processors are also changing as manufacturers are increasingly offering AI-focused machines that come with NPUs (neural processing units). What are NPUs, do users in your organisation need them, and what impact will NPUs have on virtual environments?
After all, we have only just got to the stage where effective virtualisation solutions exist that allow the creation of virtual GPUs so the resources of one physical GPU can be used on multiple virtual machines.
The rapid charge into the AI era now means you have another factor to consider – NPUs – when looking at your VDI (virtual desktop infrastructure) options.
NPU Overview
NPUs differ from CPUs (central processing units that are general purpose) and GPUs (graphics processing units that accelerate the processing of images and graphics). NPUs, on the other hand, are optimised for data-driven parallel computing. This means NPUs can perform trillions of operations a second.
NPUs are not a new technology – they have been part of the chips in Apple devices for several years. One of the main benefits of NPUs, however, is their ability to speed up AI tasks while using less power than a CPU or GPU.
By taking on AI tasks, NPU chips also free up CPUs and GPUs to deliver overall speed and performance improvements. As a general overview, a device will use its CPU for general tasks, its GPU for graphics tasks, and its NPU for AI tasks.
For many everyday functions, an NPU will complement the work of CPUs and GPUs. In a video call, for example, an NPU will look after blurring the background, freeing up the GPU to do what it does best, i.e., process the graphics. While not new, the increasingly widespread introduction of NPUs into Windows devices is important as Windows devices are significantly more common, especially in businesses.
The Impact of NPUs on Real-World Users
So far in this blog, we have mentioned two new hardware changes that will impact most Windows users over the coming months and years – the new AI key on Windows keyboards and the growing inclusion of NPUs on Windows devices. While both hardware features are in the AI space, they are different. Pressing the new AI key on a modern Windows keyboard will launch Copilot, Microsoft’s generative AI tool. In fact, the AI key is sometimes referred to as the Copilot key.
Copilot is a cloud-based app, so queries are processed in the cloud.
NPUs, on the other hand, are designed to handle local AI compute tasks, so are ideal for a range of situations, from hybrid model AI applications where some processing is performed locally to edge devices where the majority of compute requirements are local.
This brings into focus the question of whether or not NPUs are needed. Hardware vendors and AI evangelists might say NPUs are essential, but the reality is probably going to land somewhere similar to the current situation with GPUs.
There are users in most organisations that have minimal GPU requirements and those who are seriously GPU hungry, but most will fall somewhere in the middle. The same is likely to happen with NPUs, i.e., there will be some users who will need minimal local AI compute capabilities and others who will need a lot. The vast majority of users will be in between these extremes.
In other words, what we are probably dealing with is an evolution rather than a revolution both in how widespread AI will become and the technologies that power AI tools and functionality.
What About VDIs and NPUs?
At the time of writing, there are no widely used, trusted, and tested solutions that deliver the same virtualisation features on NPUs as currently exist with GPUs. Those solutions are probably not too far away, but the current reality could still influence decisions on whether you should migrate to VDI and when, especially for users anticipated to have heavy NPU requirements.
As with heavy graphics users, should users with heavy local AI compute requirements be supplied with physical machines so they don’t have to share NPU resources? Should you delay a decision on VDI migration until the position on virtual NPU options becomes clearer? Or will tapping the new AI key to fire up Copilot be sufficient for most users, with a virtual NPU solution implemented at a later date when there are more applications written for NPUs?
And what will be the cost implications when virtual NPU functionality is included in VDI solutions?
These questions will become clearer over time and with the current rapid pace of change, we probably won’t need to wait long. The one thing for certain is that the old adage about technology is as true today as it has ever been – just when you get on top of one technology (virtual GPUs in this case), another comes along.
