Hello,
This series adds virtio-gpu userptr support for ROCm native compute
contexts. The guest kernel pins an existing userspace mapping with
FOLL_LONGTERM and exposes it to the host as ordinary CREATE_BLOB
backing entries, avoiding a second shmem allocation and memcpy.
HINT_USERPTR and HINT_USERPTR_RDONLY live in blob_hints. They are
guest-only DRM ioctl selectors, not virtio CREATE_BLOB wire flags.
The device does not need a new wire flag. CREATE_BLOB only carries
the documented wire blob_flags (MAPPABLE / SHAREABLE / CROSS_DEVICE).
Patches overview:
1. Add VIRTIO_GPU_CAPSET_ROCM capability for compute workloads
2. Extend DRM UAPI with guest-only userptr hints and a userptr
address field
3. Implement core userptr functionality with page management
4. Wire blob ioctl creation to userptr objects
Tests:
- Full OPENCL CTS tests passed on ROCm 5.7.0 in V2000 platform.
- Near 70% percentage of OPENCL CTS tests passed on ROCm 7.0 W7900 platform.
- most HIP catch tests passed on ROCm 7.0 W7900 platform.
- Some AI applications enabled on ROCm 7.0 W7900 platform.
- latest ROCm 7.14 and ROCm 10 testing is ongoing.
V8 changes:
- Move USE_USERPTR / USERPTR_RDONLY from blob_flags into blob_hints
(HINT_USERPTR / HINT_USERPTR_RDONLY) so they do not occupy virtio
wire bits
- Drop the CREATE_BLOB blob_flags mask; hints never go on the wire
V7 changes:
- Mask guest-only DRM flags out of CREATE_BLOB wire blob_flags
- Clear userptr->pages after pin failure to avoid double-free
- DMA-map userptr SG only when virtio_gpu_use_dma_api() is required
- Use DMA_TO_DEVICE for USERPTR_RDONLY
- Sync userptr SG for the device on TRANSFER_TO_HOST
- Mark writable pages dirty when unpinning
- Reject USERPTR unless blob_mem is VIRTGPU_BLOB_MEM_GUEST
- Disallow PRIME export of userptr objects
- Note that CAPSET_ROCM uses ID 8 because ID 7 is taken by VIRCL
V6 changes:
- Rebase onto drm-misc-next
- Keep USE_USERPTR / USERPTR_RDONLY as guest-only DRM flags; drop the
virtio wire-header patch (5 patches down to 4)
- Fix userptr lifetime, DMA mapping, memlock accounting, alignment
checks, and PRIME SG export
- Updated corresponding cover letter and commit messages
V5 changes:
- Add VIRTIO_GPU_BLOB_FLAG_USERPTR_RDONLY definition to patch 2
- Dropped unused VIRTIO_GPU_F_RESOURCE_USERPTR feature bit in patch 2
- Included VIRTIO_GPU_BLOB_FLAG_USERPTR_RDONLY in
VIRTGPU_BLOB_FLAG_USE_MASK in patch 5
- Add check for userptr feature in patch 5 before creating userptr blob
resource
- Updated corresponding cover letter and commit messages
V4 changes:
- Renamed VIRTIO_GPU_CAPSET_HSAKMT to VIRTIO_GPU_CAPSET_ROCM
- Remove userptr feature probing cause it can reuse the guest
blob resource code path, reduce patch count from 6 to 5
- Updated corresponding commit messages
- Consolidated userptr feature detection in final patch
- Update corresponding cover letter content
V3 changes:
- Split into focused patches for easier review
- Removed complex interval tree userptr management
- Simplified resource creation without deduplication
- Added VIRTGPU_PARAM_RESOURCE_USERPTR for feature detection
- Improved UAPI documentation and error handling
- Enhanced code quality with proper cleanup paths
- Removed MMU notifier dependencies for simplicity
- Fixed resource lifecycle management issues
V2: - Split add HSAKMT context and blob userptr resource to two patches.
- Remove MMU notifier related patches, cause use not moveable user space
memory with MMU notifier is not a good idea.
- Remove HSAKMT context check when create context, let all the context
support the userptr feature.
- Remove MMU notifier related content in cover letter.
- Add more comments for patch 6 in cover letter.
Previous version:
https://lore.kernel.org/dri-devel/[email protected]/
Honglei Huang (4):
drm/virtio-gpu: Add VIRTIO_GPU_CAPSET_ROCM capability
drm/virtgpu api: add blob userptr resource
drm/virtio: implement userptr support for zero-copy memory access
drm/virtio: wire blob ioctl creation to userptr objects
drivers/gpu/drm/virtio/Makefile | 3 +-
drivers/gpu/drm/virtio/virtgpu_drv.h | 38 +++
drivers/gpu/drm/virtio/virtgpu_ioctl.c | 29 +-
drivers/gpu/drm/virtio/virtgpu_object.c | 10 +
drivers/gpu/drm/virtio/virtgpu_userptr.c | 330 +++++++++++++++++++++++
drivers/gpu/drm/virtio/virtgpu_vq.c | 33 ++-
include/uapi/drm/virtgpu_drm.h | 9 +
include/uapi/linux/virtio_gpu.h | 1 +
8 files changed, 442 insertions(+), 11 deletions(-)
create mode 100644 drivers/gpu/drm/virtio/virtgpu_userptr.c
base-commit: 766bfba0f3bae329f99b42dcabc3ef11fa368f0b
--
2.34.1
