On Thu, 22 Oct 2026 15:12:52 -0600 Cordell Bloor wrote: [...] > Hi Francesco,
Hello Cordell,
thanks a lot for your kind reply!
>
> On 2026-10-22 12:23, Francesco Poli (frx) wrote:
> > $ llama-bench -hf MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF
[...]
> > ROCm error: invalid device function
[...]
>
> The "invalid device function" error that you're seeing is the current
> ROCm way of saying "the library you're trying to use was not built
> for your GPU." The gfx90c architecture is not currently on the list
> of GPUs supported by ROCm math libraries in Unstable or Forky,
> because the "fallback architecture" patch we were using to enable
> gfx90c in ROCm 5 could not easily be carried over into ROCm 6 or
> later.
So, just to be sure I am understanding correctly: there's no variant of
libggml0-backend-hip built to target the gfx90c GPUs in the Debian
package, since the ROCm math libraries cannot (in current Debian
testing/unstable) made to support gfx90c architecture.
Does this also mean that I cannot (currently) use this integrated GPU
for GPGPU computations? I haven't yet tried...
> This was
> mentioned briefly in the NEWS for rocm-hipamd 6.4.3-1 [1]. There is a
> suggested workaround on the Supported GPU wiki page [2] (which also
> reminds me that the wiki has not yet been updated for Forky or
> current Unstable). For posterity, I'll repeat the suggestion here:
> try setting `export HSA_OVERRIDE_GFX_VERSION=9.0.0` in your
> environment to use the code objects built for gfx900 on your gfx90c
> GPU.
It works:
```
$ export HSA_OVERRIDE_GFX_VERSION=9.0.0
$ llama-bench -hf MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF
ggml_cuda_init: found 1 ROCm devices (Total VRAM: 13977 MiB):
Device 0: AMD Radeon Graphics, gfx900:xnack+ (0x900), VMM: no, Wave Size: 64,
VRAM: 13977 MiB
load_backend: loaded ROCm backend from
/usr/lib/x86_64-linux-gnu/ggml/backends0/libggml-hip.so
load_backend: loaded CPU backend from
/usr/lib/x86_64-linux-gnu/ggml/backends0/libggml-cpu-haswell.so
| model | size | params | backend | ngl |
test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: |
--------------: | -------------------: |
| llama 7B Q4_K - Medium | 4.07 GiB | 7.25 B | ROCm | -1 |
pp512 | 53.19 ± 0.06 |
| llama 7B Q4_K - Medium | 4.07 GiB | 7.25 B | ROCm | -1 |
tg128 | 6.90 ± 0.01 |
build: Debian (11669)
```
>
> I'm hoping we'll be able to get gfx90c working without needing those
> sorts of hacks for Forky, but I cannot make any hard promises. All I
> can say is that I will do my best. However, I'd be surprised if the
> HIP backend performs better than the vulkan implementation on gfx90c
> even if I get it working.
Your guess seems to be confirmed by the actual numbers.
It seems that the Vulkan backend performs better.
> I don't think upstream has done much rocblas kernel
> tuning for gfx90c (if any), but tuned kernels are vital for achieving
> optimal performance on the ROCm BLAS stack.
>
> Sincerely,
> Cory Bloor
>
> [1]:
> https://salsa.debian.org/rocm-team/rocm-hipamd/-/blob/debian/6.4.3-1/debian/NEWS?ref_type=tags
> [2]:
> https://salsa.debian.org/rocm-team/community/team-project/-/wikis/Supported-GPU-list#gfx90c
Thank you so much for your insight.
I am looking forward to hearing back from you (especially regarding my
question about GPGPU)!
Also, if you have anything to comment about my comparison of Vulkan and
CPU backends (why is `pp512` performance almost double, while `tg128`
is only very slightly better), I would be very interested in hearing
your take on that...
Thanks again for you time and patience! :-)
--
https://...?
There's not a second to spare! To the laboratory!
..................................................... Francesco Poli .
GnuPG key fpr == CA01 1147 9CD2 EFDF FB82 3925 3E1C 27E1 1F69 BFFE
pgp2jHOoUev8q.pgp
Description: OpenPGP digital signature
