At the moment sPAPR PHB supports only a single 32bit window which is normally 1..2GB which is not enough for high performance devices.
PAPR spec enables creating an additional window(s) to support 64bit DMA and bigger page sizes. This patchset adds DDW support for pseries. The host kernel changes are required, posted earlier today as: [PATCH v3 00/24] powerpc/iommu/vfio: Enable Dynamic DMA windows This was tested on POWER8 system which allows one additional DMA window which is mapped at 0x800.0000.0000.0000 and supports 16MB pages. Existing guests check for DDW capabilities in PHB's device tree and if it is present, they request for an additional window and map entire guest RAM using H_PUT_TCE/... hypercalls once at boot time and switch to direct DMA operations. TCE tables still may be big enough for guests backed with 64K pages but they are reasonably small for guests backed by 16MB pages. This does not contain PCI 64bit BAR support and VIO-TCE-bypass rework, these are required for this to work but they have been posted separately today. Please comment. Thanks! Changes: v4: * (!) reimplemented the whole thing * machine reset and ddw-reset RTAS call both remove all TCE tables and create the default one * IOMMU group id is not needed to use VFIO PHB anymore, multiple groups are supported on the same VFIO container and virtual PHB v3: * removed "reset" from API now * reworked machine versions * applied multiple comments * includes David's machine QOM rework as this patchset adds a new machine type v2: * tested on emulated PHB * removed "ddw" machine property, now it is PHB property * disabled by default * defined "pseries-2.2" machine which enables DDW by default * fixed reset() and reference counting Alexey Kardashevskiy (18): spapr_iommu: Disable in-kernel IOMMU tables for >4GB windows spapr_iommu: Make H_PUT_TCE_INDIRECT endian-safe spapr_pci: Introduce a liobn number generating macros spapr_vio: Introduce a liobn number generating macros spapr_pci: Make find_phb()/find_dev() public spapr_iommu: Make spapr_tce_find_by_liobn() public spapr_iommu: Implement free_table() helper vfio: Add DMA memory registering spapr_rtas: Reserve DDW RTAS token numbers spapr_pci: Define DDW callbacks spapr_pci/spapr_pci_vfio: Support Dynamic DMA Windows (DDW) spapr_rtas: Add Dynamic DMA windows (DDW) RTAS handlers spapr_pci: Advertise dynamic DMA windows to guest vfio: Enable DDW ioctls to VFIO IOMMU driver spapr_pci_vfio: Enable multiple groups per container spapr_rtas_ddw: Workaround broken LE guests target-ppc: kvm: make use of KVM_CREATE_SPAPR_TCE_64 vfio: Enable in-kernel acceleration via VFIO KVM device hw/ppc/Makefile.objs | 3 + hw/ppc/spapr.c | 5 + hw/ppc/spapr_iommu.c | 23 ++- hw/ppc/spapr_pci.c | 209 ++++++++++++++++++++------ hw/ppc/spapr_pci_vfio.c | 108 +++++++++----- hw/ppc/spapr_rtas.c | 29 ++++ hw/ppc/spapr_rtas_ddw.c | 337 ++++++++++++++++++++++++++++++++++++++++++ hw/ppc/spapr_vio.c | 2 +- hw/vfio/common.c | 167 ++++++++++++++++++--- include/hw/pci-host/spapr.h | 38 ++++- include/hw/ppc/spapr.h | 15 +- include/hw/vfio/vfio-common.h | 3 +- include/hw/vfio/vfio.h | 6 +- target-ppc/kvm.c | 48 ++++-- target-ppc/kvm_ppc.h | 10 +- trace-events | 4 + 16 files changed, 881 insertions(+), 126 deletions(-) create mode 100644 hw/ppc/spapr_rtas_ddw.c -- 2.0.0
