In a typical dma-buf use case, a dmabuf exporter makes its buffer
buffer available to an importer by mapping it using DMA APIs
such as dma_map_sgtable() or dma_map_resource(). However, this
is not desirable in some cases where the exporter and importer
are directly connected via a physical or virtual link (or
interconnect) and the importer can access the buffer without
having it DMA mapped.
So, to address this scenario, this patch series adds APIs to map/
unmap dmabufs via interconnects and also provides a helper to
identify the first common interconnect between the exporter and
importer. Furthermore, this patch series also adds support for
IOV interconnect in the vfio-pci driver and Intel Xe driver.
The IOV interconnect is a virtual interconnect between an SRIOV
physical function (PF) and its virtual functions (VFs). And, for
the IOV interconnect, the addresses associated with a buffer are
shared using an xarray (instead of an sg_table) that is populated
with entries of type struct range.
The dma-buf patches in this series are based on ideas/suggestions
provided by Jason Gunthorpe, Christian Koenig and Thomas Hellström.
Changelog:
RFC -> RFCv2:
- Add documentation for the new dma-buf APIs and types (Thomas)
- Change the interconnect type from enum to unique pointer (Thomas)
- Moved the new dma-buf APIs to a separate file
- Store a copy of the interconnect matching data in the attachment
- Simplified the macros to create and match interconnects
- Use struct device instead of struct pci_dev in match data
- Replace DRM_INTERCONNECT_DRIVER with XE_INTERCONNECT_VRAM during
address encoding (Matt, Thomas)
- Drop is_devmem_external and instead rely on bo->dma_data.dma_addr
to check for imported VRAM BOs (Matt)
- Pass XE_PAGE_SIZE as the last parameter to xe_bo_addr (Matt)
- Add a check to prevent malicious VF from accessing other VF's
addresses (Thomas)
- Fallback to legacy (map_dma_buf) mapping method if mapping via
interconnect fails
Patchset overview:
Patch 1-3: Add dma-buf APIs to map/unmap and match
Patch 4: Add support for IOV interconnect in vfio-pci driver
Patch 5: Add support for IOV interconnect in Xe driver
Patch 6-8: Create and use a new dma_addr array for LMEM based
dmabuf BOs to store translated addresses (DPAs)
This series is rebased on top of the following repo:
https://git.kernel.org/pub/scm/linux/kernel/git/leon/linux-rdma.git/log/?h=dmabuf-vfio-v5
Associated Qemu patch series:
https://lore.kernel.org/qemu-devel/[email protected]/
Associated vfio-pci patch series:
https://lore.kernel.org/dri-devel/[email protected]/
This series is tested using the following method:
- Run Qemu with the following relevant options:
qemu-system-x86_64 -m 4096m ....
-device ioh3420,id=root_port1,bus=pcie.0
-device x3130-upstream,id=upstream1,bus=root_port1
-device xio3130-downstream,id=downstream1,bus=upstream1,chassis=9
-device xio3130-downstream,id=downstream2,bus=upstream1,chassis=10
-device vfio-pci,host=0000:03:00.1,bus=downstream1
-device virtio-gpu,max_outputs=1,blob=true,xres=1920,yres=1080,bus=downstream2
-display gtk,gl=on
-object memory-backend-memfd,id=mem1,size=4096M
-machine q35,accel=kvm,memory-backend=mem1 ...
- Run Gnome Wayland with the following options in the Guest VM:
# cat /usr/lib/udev/rules.d/61-mutter-primary-gpu.rules
ENV{DEVNAME}=="/dev/dri/card1", TAG+="mutter-device-preferred-primary",
TAG+="mutter-device-disable-kms-modifiers"
# XDG_SESSION_TYPE=wayland dbus-run-session -- /usr/bin/gnome-shell --wayland
--no-x11 &
Cc: Jason Gunthorpe <[email protected]>
Cc: Leon Romanovsky <[email protected]>
Cc: Christian Koenig <[email protected]>
Cc: Sumit Semwal <[email protected]>
Cc: Thomas Hellström <[email protected]>
Cc: Simona Vetter <[email protected]>
Cc: Matthew Brost <[email protected]>
Cc: Dongwon Kim <[email protected]>
Vivek Kasireddy (8):
dma-buf: Add support for map/unmap APIs for interconnects
dma-buf: Add a helper to match interconnects between exporter/importer
dma-buf: Create and expose IOV interconnect to all exporters/importers
vfio/pci/dmabuf: Add support for IOV interconnect
drm/xe/dma_buf: Add support for IOV interconnect
drm/xe/pf: Add a helper function to get a VF's backing object in LMEM
drm/xe/bo: Create new dma_addr array for dmabuf BOs associated with
VFs
drm/xe/pt: Add an additional check for dmabuf BOs while doing bind
drivers/dma-buf/Makefile | 2 +-
drivers/dma-buf/dma-buf-interconnect.c | 164 +++++++++++++++++++++
drivers/dma-buf/dma-buf.c | 12 +-
drivers/gpu/drm/xe/xe_bo.c | 162 ++++++++++++++++++--
drivers/gpu/drm/xe/xe_bo_types.h | 6 +
drivers/gpu/drm/xe/xe_dma_buf.c | 17 ++-
drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c | 24 +++
drivers/gpu/drm/xe/xe_gt_sriov_pf_config.h | 1 +
drivers/gpu/drm/xe/xe_pt.c | 8 +
drivers/gpu/drm/xe/xe_sriov_pf_types.h | 19 +++
drivers/vfio/pci/vfio_pci_dmabuf.c | 135 ++++++++++++++++-
include/linux/dma-buf-interconnect.h | 122 +++++++++++++++
include/linux/dma-buf.h | 41 ++++++
13 files changed, 691 insertions(+), 22 deletions(-)
create mode 100644 drivers/dma-buf/dma-buf-interconnect.c
create mode 100644 include/linux/dma-buf-interconnect.h
--
2.50.1
