Currently, the steering tag can be used for a CPU on the motherboard; the ACPI check is in place to query and obtain the supported tph settings. Here we intend to use the tph info to improve RDMA NIC memory access on a vfio-based accelerator device via PCIe peer-to-peer. When an application registers a RDMA memory region with DMABUF for the RDMA NIC to access the device memory, the tph info of the memory region can be retrieved and used to set the steering tag / process hint (ph). Additional instructions or hints can be passed to the GPU or accelerator device for advanced memory operations, such as, read cache selection.
Note this RFC is for the discussion on the direction and is not intended to be a complete implementation. If no objection, we will convert it to a Patch set. Changes v1 -> v2: - Encode steering tag and ph in vfio_device_feature_dma_buf flags field instead of adding new uapi struct fields, to preserve ABI compatibility - Fix subject prefixes: "Vfio:" -> "vfio:", "RMDA MLX5:" -> "RDMA/mlx5:" - Fix raw steering tag vs st_index mismatch: get_tph() now returns a raw steering tag, and the mlx5 consumer converts it to an st_index via the new mlx5_st_alloc_index_by_tag() API - Fix @tph doc typo to @steering_tag in dma-buf.h - Remove unused variable, fix parameter alignment, fix trailing semicolon Zhiping Zhang (2): vfio: add callback to get tph info for dmabuf RDMA/mlx5: get tph for p2p access when registering dmabuf mr drivers/infiniband/hw/mlx5/mr.c | 34 +++++++++++++++++++ .../net/ethernet/mellanox/mlx5/core/lib/st.c | 23 +++++++++---- drivers/vfio/pci/vfio_pci_dmabuf.c | 26 ++++++++++++-- include/linux/dma-buf.h | 30 ++++++++++++++++ include/linux/mlx5/driver.h | 7 ++++ include/uapi/linux/vfio.h | 9 +++-- 6 files changed, 118 insertions(+), 11 deletions(-) -- 2.52.0
