This series can be found on GitHub: https://github.com/dmatlack/linux/tree/liveupdate/vfio/cdev/v2
This series adds the base support to preserve a VFIO device file across a Live Update. "Base support" means that this allows userspace to safely preserve a VFIO device file with LIVEUPDATE_SESSION_PRESERVE_FD and retrieve it with LIVEUPDATE_SESSION_RETRIEVE_FD, but the device itself is not preserved in a fully running state across Live Update. This series aims to provide a foundation on which to build the rest of the device preservation infrastructure, including: - Preservation of iommufd files [1] - Preservation of IOMMU driver state - Preservation of PCI state (BAR resources, device state, bridge state, ...) - Preservation of vfio-pci driver state Testing ------- The patches at the end of this series provide comprehensive selftests for the new code added by this series. The selftests have been validated in both a VM environment using a virtio-net PCIe device, and in a baremetal environment on an Intel EMR server with an Intel DSA PCIe device. Here is an example of how to run the new selftests: vfio_pci_liveupdate_uapi_test: $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test 0000:00:04.0 $ tools/testing/selftests/vfio/scripts/cleanup.sh vfio_pci_liveupdate_kexec_test: $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 1 0000:00:04.0 $ kexec ... $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 2 0000:00:04.0 $ tools/testing/selftests/vfio/scripts/cleanup.sh It is also possible to run vfio_pci_liveupdate_kexec_test multiple times to preserve multiple devices simultaneously across a Live Update. This series has been tested with up to 8 devices concurrently preserved. Changelog --------- v2: - Rebase on top of linux-next (tag: next-20260115) - Add missing EXPORT_SYMBOL_GPLs in LUO (Zhu) - Add Missing EXPORT_SYMBOL_GPLs for vfio_device_fops (Zhu) - Fix circular dependency between vfio-pci-core and vfio-pci (Zhu) - Handle pci=assign-busses (Lukas) - Drop driver_override patch (Jason) - Use kho_alloc_preserve(), kho_unpreserve_free(), kho_restore_free() (Pasha) - Don't access PCI FLB after device initialization (Jason) - Fix folio leak in vfio_pci_liveupdate_retrieve() (Alex) - Add Documentation (Pasha) v1: https://lore.kernel.org/kvm/[email protected]/ rfc: https://lore.kernel.org/kvm/[email protected]/ Cc: Saeed Mahameed <[email protected]> Cc: Adithya Jayachandran <[email protected]> Cc: Jason Gunthorpe <[email protected]> Cc: Parav Pandit <[email protected]> Cc: Leon Romanovsky <[email protected]> Cc: William Tu <[email protected]> Cc: Jacob Pan <[email protected]> Cc: Lukas Wunner <[email protected]> Cc: Pasha Tatashin <[email protected]> Cc: Mike Rapoport <[email protected]> Cc: Pratyush Yadav <[email protected]> Cc: Samiullah Khawaja <[email protected]> Cc: Chris Li <[email protected]> Cc: Josh Hilke <[email protected]> Cc: David Rientjes <[email protected]> [1] https://lore.kernel.org/linux-iommu/[email protected]/ David Matlack (13): liveupdate: Export symbols needed by modules PCI: Add API to track PCI devices preserved across Live Update PCI: Inherit bus numbers from previous kernel during Live Update vfio/pci: Notify PCI subsystem about devices preserved across Live Update vfio: Enforce preserved devices are retrieved via LIVEUPDATE_SESSION_RETRIEVE_FD vfio/pci: Store incoming Live Update state in struct vfio_pci_core_device docs: liveupdate: Document VFIO device file preservation vfio: selftests: Add Makefile support for TEST_GEN_PROGS_EXTENDED vfio: selftests: Add vfio_pci_liveupdate_uapi_test vfio: selftests: Expose iommu_modes to tests vfio: selftests: Expose low-level helper routines for setting up struct vfio_pci_device vfio: selftests: Verify that opening VFIO device fails during Live Update vfio: selftests: Add continuous DMA to vfio_pci_liveupdate_kexec_test Vipin Sharma (9): vfio/pci: Register a file handler with Live Update Orchestrator vfio/pci: Preserve vfio-pci device files across Live Update vfio/pci: Retrieve preserved device files after Live Update vfio/pci: Skip reset of preserved device after Live Update selftests/liveupdate: Move luo_test_utils.* into a reusable library selftests/liveupdate: Add helpers to preserve/retrieve FDs vfio: selftests: Build liveupdate library in VFIO selftests vfio: selftests: Initialize vfio_pci_device using a VFIO cdev FD vfio: selftests: Add vfio_pci_liveupdate_kexec_test Documentation/userspace-api/liveupdate.rst | 144 ++++++++++ MAINTAINERS | 1 + drivers/pci/Makefile | 1 + drivers/pci/liveupdate.c | 212 +++++++++++++++ drivers/pci/probe.c | 23 +- drivers/vfio/device_cdev.c | 25 +- drivers/vfio/group.c | 9 + drivers/vfio/pci/Makefile | 1 + drivers/vfio/pci/vfio_pci.c | 11 +- drivers/vfio/pci/vfio_pci_core.c | 24 +- drivers/vfio/pci/vfio_pci_liveupdate.c | 246 +++++++++++++++++ drivers/vfio/pci/vfio_pci_priv.h | 16 ++ drivers/vfio/vfio.h | 13 - drivers/vfio/vfio_main.c | 23 +- include/linux/kho/abi/pci.h | 55 ++++ include/linux/kho/abi/vfio_pci.h | 45 +++ include/linux/pci.h | 47 ++++ include/linux/vfio.h | 58 ++++ include/linux/vfio_pci_core.h | 2 + kernel/liveupdate/luo_core.c | 1 + kernel/liveupdate/luo_file.c | 2 + tools/testing/selftests/liveupdate/.gitignore | 1 + tools/testing/selftests/liveupdate/Makefile | 14 +- .../include/libliveupdate.h} | 11 +- .../selftests/liveupdate/lib/libliveupdate.mk | 20 ++ .../{luo_test_utils.c => lib/liveupdate.c} | 43 ++- .../selftests/liveupdate/luo_kexec_simple.c | 2 +- .../selftests/liveupdate/luo_multi_session.c | 2 +- tools/testing/selftests/vfio/Makefile | 23 +- .../vfio/lib/include/libvfio/iommu.h | 2 + .../lib/include/libvfio/vfio_pci_device.h | 8 + tools/testing/selftests/vfio/lib/iommu.c | 4 +- .../selftests/vfio/lib/vfio_pci_device.c | 60 ++-- .../vfio/vfio_pci_liveupdate_kexec_test.c | 256 ++++++++++++++++++ .../vfio/vfio_pci_liveupdate_uapi_test.c | 93 +++++++ 35 files changed, 1410 insertions(+), 88 deletions(-) create mode 100644 drivers/pci/liveupdate.c create mode 100644 drivers/vfio/pci/vfio_pci_liveupdate.c create mode 100644 include/linux/kho/abi/pci.h create mode 100644 include/linux/kho/abi/vfio_pci.h rename tools/testing/selftests/liveupdate/{luo_test_utils.h => lib/include/libliveupdate.h} (80%) create mode 100644 tools/testing/selftests/liveupdate/lib/libliveupdate.mk rename tools/testing/selftests/liveupdate/{luo_test_utils.c => lib/liveupdate.c} (89%) create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c base-commit: 9b7977f9e39b7768c70c2aa497f04e7569fd3e00 -- 2.53.0.rc1.225.gd81095ad13-goog
