Hi all,
ChatGPT[1] has been popular for several years, perhaps you also know
Deepseek[2]
which is an open-source language model and run locally.
The local runner to run Deepseek is Ollama[3]. It lets you download and run
large language models (LLMs) on your own machine, without relying on
cloud-hosted services.
That is good for Yocto, so I integrate ollama to Yocto-based Linux system,
build from source, offline install and deploy.
For the convenience of review, show it on my github [4] temporary,
- The BUILD.md [5] provides the steps to build and run on CPU by default,
- The BUILD-cuda-x86-64.md [6] provides the steps to build and run on
NVIDIA GPU
with CUDA for x86-64
And now contribute meta-ollama to Yocto, the layer provides:
- Recipe ollama: provides application ollama.
It supports to run on CPU by default. And optional on NVIDIA GPU with
CUDA
- Recipe llama3.2-1b and llama3.2-3b: the large language model by Meta [7]
- Recipe gemma2-9b and gemma2-2b: the large language model by Google [8]
- Recipe deepseek-r1-7b and deepseek-r1-1dot5b, the large language model
by Deepseek [9]
- Recipe nvidia-open-gpu-kernel-module: the kernel module for NVIDIA GPU
- Recipe nvidia-driver-x86-64: provides firmware for NVIDIA kernel module,
CUDA library and application nvidia-smi for x86-64 BSP. NOTE, no sources,
only binaries from NVIDIA [10], license is Proprietary [11]
- dynamic-layers/meta-tegra: customize meta-tegra to support CUDA for x86-64
Hi Richard,
Would you please approve to add this layer to
https://git.yoctoproject.org/, I will maintain
meta-ollama for recipe uprev and bug fix
[1] https://chatgpt.com/
[2] https://www.deepseek.com/
[3] https://ollama.com/
[4] https://github.com/hongxu-jia/meta-ollama
[5] https://github.com/hongxu-jia/meta-ollama/blob/main/BUILD.md
[6] https://github.com/hongxu-jia/meta-ollama/blob/main/BUILD-cuda-x86-64.md
[7] https://ollama.com/library/llama3.2
[8] https://ollama.com/library/gemma2
[9] https://ollama.com/library/deepseek-r1
[10]
https://developer.download.nvidia.com/compute/nvidia-driver/redist/nvidia_driver/linux-x86_64/
[11]
https://developer.download.nvidia.com/compute/nvidia-driver/redist/redistrib_590.48.01.json
//Hongxu
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#2219):
https://lists.openembedded.org/g/openembedded-architecture/message/2219
Mute This Topic: https://lists.openembedded.org/mt/117540395/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
