Tested following intel recommandation to run Qwen/Qwen2-7B model in LNL based
on
https://docs.openvino.ai/2024/learn-openvino/llm_inference_guide/genai-guide-npu.html
using latest Linux NPU driver -
https://github.com/intel/linux-npu-driver/releases/tag/v1.10.1
:
$ uname -a
Linux tucano-806928 6.8.0-57-generic
$ sudo -E npu-env/bin/python
>>> import openvino_genai as ov_genai
>>> model_path="Llama-2-7B-Chat-GPTQ"
>>> pipe = ov_genai.LLMPipeline(model_path, "NPU")
>>> print(pipe.generate("The Sun is yellow because", max_new_tokens=100))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:223:
Exception from
src/plugins/intel_npu/src/plugin/npuw/just_sync_infer_request.cpp:659:
Failed to compile. No more devices are left!
======
$ uname -a
Linux tucano-806928 6.8.0-58-generic #60-Ubuntu
$ sudo -E npu-env/bin/python
Python 3.12.3 (main, Feb 4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import openvino_genai as ov_genai
>>> pipe = ov_genai.LLMPipeline("Llama-2-7B-Chat-GPTQ", "NPU")
>>> print(pipe.generate("The Sun is yellow because", max_new_tokens=100))
of the way it absorbs and scatters light.
The Sun appears yellow to us because of the way it absorbs and scatters light.
When light from the Sun hits an object, some of it is absorbed and some of it
is scattered in all directions. The amount of light that is scattered depends
on the color of the object and the wavelength of the light.
The Sun absorbs light in the yellow and orange parts of the spectrum, which is
why it
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2098972
Title:
Enable Large Language Model (LLM) workloads using Intel NPU
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2098972/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
