This small series adds support for Dirty Tracking in IOMMUFD backend.
The sole reason I still made it RFC is because of the second patch,
where we are implementing user-managed auto domains.
In essence it is quite similar to the original IOMMUFD series where we
would allocate a HWPT, until we switched later on into a IOAS attach.
Patch 2 goes into more detail, but the gist is that there's two modes of
using IOMMUFD and by keep using kernel managed auto domains we would end
up duplicating the same flags we have in HWPT but into the VFIO IOAS
attach. While it is true that just adding a flag is simpler, it also
creates duplication and motivates duplicate what hwpt-alloc already has.
But there's a chance I have the wrong expectation here, so any feedback
welcome.
The series is divided into:
* Patch 1: Adds a simple helper to get device capabilities;
* Patches 2 - 5: IOMMUFD backend support for dirty tracking;
The workflow is relatively simple:
1) Probe device and allow dirty tracking in the HWPT
2) Toggling dirty tracking on/off
3) Read-and-clear of Dirty IOVAs
The heuristics selected for (1) were to enable it *if* device supports
migration but doesn't support VF dirty tracking or IOMMU dirty tracking
is supported. The latter is for the hotplug case where we can add a device
without a tracker and thus still support migration.
The unmap case is deferred until further vIOMMU support with migration
is added[3] which will then introduce the usage of
IOMMU_HWPT_GET_DIRTY_BITMAP_NO_CLEAR in GET_DIRTY_BITMAP ioctl in the
dma unmap bitmap flow.
* Patches 6-8: Add disabling of hugepages to allow tracking at base
page; avoid blocking live migration where there's no VF dirty
tracker, considering that we have IOMMU dirty tracking. And allow
disabling VF dirty tracker via qemu command line.
This series builds on top of Zhengzhong series[0], but only requires the
first 9 patches i.e. up to ("vfio/pci: Initialize host iommu device
instance after attachment")[1] that are more generic IOMMUFD device
plumbing, and doesn't require the nesting counterpart.
This is stored on github:
https://github.com/jpemartins/qemu/commits/iommufd-v5
Note: While Linux v6.7 has IOMMU dirty tracking feature, I suggest folks
use the latest for-rc of iommufd kernel tree as there's some fixes there.
Comments and feedback appreciated.
Cheers,
Joao
Chances since RFCv1[2]:
* Remove intel/amd dirty tracking emulation enabling
* Remove the dirtyrate improvement for VF/IOMMU dirty tracking
[Will pursue these two in separate series]
* Introduce auto domains support
* Enforce dirty tracking following the IOMMUFD UAPI for this
* Add support for toggling hugepages in IOMMUFD
* Auto enable support when VF supports migration to use IOMMU
when it doesn't have VF dirty tracking
* Add a parameter to toggle VF dirty tracking
[0]
https://lore.kernel.org/qemu-devel/20240201072818.327930-1-zhenzhong.d...@intel.com/
[1]
https://lore.kernel.org/qemu-devel/20240201072818.327930-10-zhenzhong.d...@intel.com/
[2]
https://lore.kernel.org/qemu-devel/20220428211351.3897-1-joao.m.mart...@oracle.com/
[3]
https://lore.kernel.org/qemu-devel/20230622214845.3980-1-joao.m.mart...@oracle.com/
Joao Martins (8):
backends/iommufd: Introduce helper function
iommufd_device_get_hw_capabilities()
vfio/iommufd: Introduce auto domain creation
vfio/iommufd: Probe and request hwpt dirty tracking capability
vfio/iommufd: Implement VFIOIOMMUClass::set_dirty_tracking support
vfio/iommufd: Implement VFIOIOMMUClass::query_dirty_bitmap support
backends/iommufd: Add ability to disable hugepages
vfio/migration: Don't block migration device dirty tracking is
unsupported
vfio/common: Allow disabling device dirty page tracking
backends/iommufd.c | 133 ++++++++++++++++++++++++++++
backends/trace-events | 4 +
hw/vfio/common.c | 32 ++++++-
hw/vfio/iommufd.c | 162 ++++++++++++++++++++++++++++++++++
hw/vfio/migration.c | 5 +-
hw/vfio/pci.c | 3 +
include/hw/vfio/vfio-common.h | 12 +++
include/sysemu/iommufd.h | 17 ++++
qapi/qom.json | 2 +-
9 files changed, 367 insertions(+), 3 deletions(-)
--
2.39.3
