On 8/17/2026 5:06 PM, Alex Bennée wrote:
"Huang, Honglei" <[email protected]> writes:
Hi Michael, Alex, Dmitry, Akihiko,
I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm
over virtio
implementation to virglrenderer nine months ago (MR !1568 [1]). The
ROCm side has
been supportted by ROCm offical.
Current implementation is a virtio gpu context type capset handled inside
virglrenderer, sharing the display path. That's an awkward fit, many
compute GPUs have no display engine at all.
Beyond that, sharing the display path is increasingly painful:
- Compute hammers the queues more than graphics, so sharing
virtio gpu's single control queue with display/virgl causes contention
and display stutter.
Is this just due to iteration? From a layman's point of view I'm curious
as to what the queues are doing.
Really thanks for the reply.
The load is mostly memory management. Running an AI model allocates and
frees a large number of blobs. We already did some optimization release
them asynchronously, but the host processing is a single queue one
process_cmdq, this is where the main bottleneck in my debugging work /
my understanding so far. I'm not certain it's the whole picture, so
please correct if I am wrong.
A model load or unload frees a large batch of BOs and allocates another.
Some of those commands are async in the virtio-gpu guest driver, but
QEMU still has to work through them on the one queue, which takes time;
so even though any single command is quick, there are simply too many of
them, the single queue backs up, and everything behind it, gets delayed.
real work load (a few downstream customisations): loading one 16 GB
model (gemm4 e4b), drives ~1200 blob creates, a burst of
~1400 resource frees at teardown, ~3700 submits and ~6000 virtqueue
notifies, caused a 22 s guest soft lockup. And the behavior of memory
operations are controlled by upper layer like pytorch / HIP / runtime,
we can not control it.
To be honest, a separate backend won't fix this. But the real solution
maybe is compute specific. That logic is only useful to the compute
path, and folding it into the shared display device / renderer would
mean churning code that is mature and stable for graphics, with
regression risk. Keeping compute on its own instance and backend lets us
iterate on these compute only optimisations.
- Compute contexts need far more blob / shared memory than a display
one.
I guess weights and context are long lived blobs compared to rendering assets?
Exactly, weights, KV caches, scratch heaps and the userptr/SVM arenas
are large and long-lived, unlike per frame rendering assets, so an AI
context maps far more memory than a display one.
- Maybe needs a wider ROCm / compute stack, cause the render model
fits poorly:
rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth streams)
and ROCgdb (wave control, address watch, async exceptions an
out of band channel that must not block display).
- Events, faults and GPU reset/SMI are async and don't map onto fences.
- All of this is hard to extend cleanly inside a display capset.
On the QEMU/host side, would something like this be OK? One step, two parts:
- a dedicated headless virtio gpu instance for compute.
An oft-asked for VirtIO model is virtio-npu but I wonder if there is
enough commonality for a virtio-compute device with the same sort native
context type handling to deal with the those that want to have guests
targeting specific hardware rather than going through an abstraction
like Vulkan Computer or OpenGL CL.
Agreed there's likely enough commonality. The idea would be for the
device to provide mechanism, not hardware policy. And actually the
current ROCm implementation is doing like this, it can also support
OPENCL and HIP.
- that instance served by a separate ROCm backend library loaded
in-process by QEMU.
Is this library a binary blob or open source? The last time I looked at
ROCm I had to give up as my AMD card was in an Aarch64 AVA machine.
It is open source, we are planing put it into ROCm stack, let it be a
part of ROCm.
Regards,
Honglei
That reuses the existing pluggable backend model, a second virtio gpu + a
backend library. It doesn't add dedicated queues for debug/profiling
currently.
Waiting for reply and happy to share more detail. Thanks!
[1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests/1568
Regards,
Honglei
