This series is a follow up to the earlier work in the drm_gpusvm_pages
decoupling series [1], and follows the direction Matt suggested [2].
drm_gpusvm_get_pages() used to do two things at once: the MM level HMM
fault of the CPU range, and the device DMA mapping of the faulted pages.
When one CPU range is mirrored on several devices, each device had
to call get_pages() and redo the HMM fault every time. Matt suggested [2]
passing an array of drm_gpusvm_pages plus a count so the fault is done
once and shared, while keeping the notifier retry loop in common code so
drivers never open code it.
The series builds up to that in small steps, no functional change until
the last two patches add the new behaviour:
- patch 1 pulls the HMM fault, and its -EBUSY retry, into
drm_gpusvm_hmm_fault() so it can be shared.
- patch 2 moves the dma_addr allocation out of the notifier locked
section, so the mapping step becomes self contained.
- patch 3 pulls the per-device mapping loop into
drm_gpusvm_dma_map_pages().
- patch 4 makes get_pages() take an array of drm_gpusvm_pages plus a
count: fault once, then DMA map each instance, one per owning
drm_device, under a single notifier retry gate. The common 1:1 case
passes count == 1.
- patch 5 adds no_dma_map so a driver that only needs
the CPU pages faulted in (e.g. AMDXDNA using GPU SVM for userptr)
can skip the device DMA mapping.
tests:
AMDGPU:
SVM:DRM N:1 multi device support is work in progress on top of this
series. The single device (1:1) path was tested with the amdgpu SVM
adaptation on top. Based on amdgpu SVM [3].
Tested on gfx943 (MI300X) and gfx906 (MI60) with XNACK on/off:
- KFD test: 95%+ passed.
- ROCR test: all passed.
- HIP catch test: gfx943 (MI300X): 99% passed.
gfx906 (MI60): 99% passed.
links:
[1] drm_gpusvm_pages decoupling series:
https://lore.kernel.org/amd-gfx/[email protected]/
[2] Matt's suggested direction:
https://lore.kernel.org/amd-gfx/[email protected]/#:~:text=Hmm%2C%20this%20might,requires%20DMA%20mapping.
[3] amdgpu SVM:
https://lore.kernel.org/amd-gfx/[email protected]/
Honglei Huang (5):
drm/gpusvm: extract drm_gpusvm_hmm_fault() helper
drm/gpusvm: move dma_addr allocation before the notifier lock
drm/gpusvm: extract drm_gpusvm_dma_map_pages() helper
drm/gpusvm: let drm_gpusvm_get_pages() map an array of pages
drm/gpusvm: make the DMA mapping step in get_pages() optional
drivers/gpu/drm/drm_gpusvm.c | 339 ++++++++++++++++++++++----------
drivers/gpu/drm/xe/xe_svm.c | 2 +-
drivers/gpu/drm/xe/xe_userptr.c | 2 +-
include/drm/drm_gpusvm.h | 6 +
4 files changed, 244 insertions(+), 105 deletions(-)
base-commit: 04984fcdbf6876c940c01026a7404c1e9cc91ba7
--
2.34.1
