The dma-map IOVA alloc, link, and sync APIs perform significantly better
than dma-map / dma-unmap, as they avoid costly IOMMU synchronizations.
This difference is especially noticeable when mapping a 2MB region in
4KB pages.
Use dma-map IOVA alloc, link, and sync APIs for GPU SVM and DRM page,
which mappings between the CPU and GPU.
Initial results are promising.
Baseline CPU time during 2M / 64K fault with a migration:
Average migrate 2M cpu time (us, percentage): 552.36049107142857142857,
.71943789893868318799
Average migrate 64K cpu time (us, percentage): 24.97767857142857142857,
.34789908128526791960
After patch 4 in this series CPU time during 2M / 64K fault with a migration:
Average migrate 2M cpu time (us, percentage): 224.81808035714285714286,
.51412827364772602557
Average migrate 64K cpu time (us, percentage): 14.65625000000000000000,
.25659463050529524405
After patch 5 in this series CPU time during 2M / 64K fault with a migration:
Average migrate 2M cpu time (us, percentage): 158.52901785714285714286,
.42740093699505625280
Average migrate 64K cpu time (us, percentage): 14.00000000000000000000,
.23276613905846987437
- Patches 2-4 implement dma-map IOVA API for system pages
- Patch 5 optimizes this a bit but questionable if we need if this
needed if 2M pages are enabled
- Patches 6-11 implement a IOVA API for multi-GPU PoC. I'm told drivers
shouldn't call dma-map IOVA API for multi-device but including it as
reference for a design we'd like to get to with high-speed fabrics in
mind.
Still an RFC as patches 6-11 may not be able to merged in there current
form, the series can be split if needed to merge eariler patches.
Matt
v2:
- Include missing basline patch for CI
v3:
- Fix memory corruption
- PoC IOVA alloc for multi-GPU
Francois Dugast (1):
drm/pagemap: Add helper to access zone_device_data
Matthew Brost (10):
drm/gpusvm: Use dma-map IOVA alloc, link, and sync API in GPU SVM
drm/pagemap: Split drm_pagemap_migrate_map_pages into device / system
drm/pagemap: Use dma-map IOVA alloc, link, and sync API for DRM
pagemap
drm/pagemap: Reduce number of IOVA link calls
drm/pagemap: Add IOVA interface to DRM pagemap
drm/xe: Stub out DRM pagemap IOVA alloc implementation
drm/pagemap: Use device-to-device IOVA alloc, link, and sync API for
DRM pagemap
drm/xe: Drop BO dma-resv lock during SVM migrate-to-device
drm/xe: Implement DRM pagemap IOVA vfuncs
drm/gpusvm: Use device-to-device IOVA alloc, link, and sync API in GPU
SVM
drivers/gpu/drm/drm_gpusvm.c | 196 ++++++++++++++++---
drivers/gpu/drm/drm_pagemap.c | 351 +++++++++++++++++++++++++++-------
drivers/gpu/drm/xe/xe_svm.c | 143 +++++++++++++-
include/drm/drm_gpusvm.h | 5 +
include/drm/drm_pagemap.h | 101 ++++++++++
5 files changed, 693 insertions(+), 103 deletions(-)
--
2.34.1
