Intel NPU Driver Preparing Hardware Scheduler & Profiling Support

[phoronix]

Phoronix: Intel NPU Driver Preparing Hardware Scheduler & Profiling Support

The Intel iVPU accelerator driver changes for the upcoming Linux 6.10 merge window have been submitted for advancing the Neural Processing Unit (NPU) support found since the launch of Meteor Lake with Intel Core Ultra notebook CPUs. For this iVPU/NPU driver in Linux 6.10 are a few notable new features...

https://www.phoronix.com/news/Intel-iVPU-Driver-NPU-Linux-610

[Eirikr1848]

Call me when it can automatically install Gentoo install and make me look super dapper and handsome on Discord and Teams calls. 💁♂️💁♂️💁♂️.

Really though, is this of any use yet for Stable Diffusion and for local LLMs?

[aviallon]

I am wondering, can this be used using Mesa's Teflon?

[hacker1024]

Call me when it can automatically install Gentoo install and make me look super dapper and handsome on Discord and Teams calls. 💁♂️💁♂️💁♂️.

Really though, is this of any use yet for Stable Diffusion and for local LLMs?

You can run Stable Diffusion and a few other things on it with Intel's GIMP plugins. It's a little disappointing: The integrated GPU is faster. You can use both together, though it still doesn't come anywhere close to my desktop RTX 3060 12GB.

Of course, the main benefit is power savings for features you'd use on the go, like AI webcam effects and things of that nature.

[tenchrio]

Really though, is this of any use yet for Stable Diffusion and for local LLMs?

It seems the Intel NPU acceleration library works at least on smaller LLMs like TinyLlama.
VPU should already accelerate Stablediffusion as they are mentioned on the torch-compile page in the execution diagram but on the same page aren't explicitly mentioned on the "Support for Automatic1111 Stable Diffusion WebUI" section so hard to say.

[brunosalezze]

This should kill the need for any gpu in DIY nvr systems. A coral TPU only has 4tops and very limited memory and people use fine up to 10 cameras. With 20 tops and power this low, face recognition with more accurate models will scale really well.