Hi! Forgot to mention - this is RFC only as 1) it depends on the host kernel changes 2) does not support migration for emulated PHB + DDW, I think I have to give David's idea with TCE table stub object a go.
Thanks On 01/29/2015 08:27 PM, Alexey Kardashevskiy wrote: > At the moment sPAPR PHB supports only a single 32bit window > which is normally 1..2GB which is not enough for high performance devices. > > PAPR spec enables creating an additional window(s) to support 64bit > DMA and bigger page sizes. > > This patchset adds DDW support for pseries. The host kernel changes are > required, posted earlier today as: > [PATCH v3 00/24] powerpc/iommu/vfio: Enable Dynamic DMA windows > > This was tested on POWER8 system which allows one additional DMA window > which is mapped at 0x800.0000.0000.0000 and supports 16MB pages. > Existing guests check for DDW capabilities in PHB's device tree and if it > is present, they request for an additional window and map entire guest RAM > using H_PUT_TCE/... hypercalls once at boot time and switch to direct DMA > operations. > > TCE tables still may be big enough for guests backed with 64K pages but they > are reasonably small for guests backed by 16MB pages. > > > This does not contain PCI 64bit BAR support and VIO-TCE-bypass rework, these > are required for this to work but they have been posted separately today. > > > Please comment. Thanks! > > Changes: > v4: > * (!) reimplemented the whole thing > * machine reset and ddw-reset RTAS call both remove all TCE tables and > create the default one > * IOMMU group id is not needed to use VFIO PHB anymore, multiple groups > are supported on the same VFIO container and virtual PHB > > v3: > * removed "reset" from API now > * reworked machine versions > * applied multiple comments > * includes David's machine QOM rework as this patchset adds a new machine type > > v2: > * tested on emulated PHB > * removed "ddw" machine property, now it is PHB property > * disabled by default > * defined "pseries-2.2" machine which enables DDW by default > * fixed reset() and reference counting > > > > > Alexey Kardashevskiy (18): > spapr_iommu: Disable in-kernel IOMMU tables for >4GB windows > spapr_iommu: Make H_PUT_TCE_INDIRECT endian-safe > spapr_pci: Introduce a liobn number generating macros > spapr_vio: Introduce a liobn number generating macros > spapr_pci: Make find_phb()/find_dev() public > spapr_iommu: Make spapr_tce_find_by_liobn() public > spapr_iommu: Implement free_table() helper > vfio: Add DMA memory registering > spapr_rtas: Reserve DDW RTAS token numbers > spapr_pci: Define DDW callbacks > spapr_pci/spapr_pci_vfio: Support Dynamic DMA Windows (DDW) > spapr_rtas: Add Dynamic DMA windows (DDW) RTAS handlers > spapr_pci: Advertise dynamic DMA windows to guest > vfio: Enable DDW ioctls to VFIO IOMMU driver > spapr_pci_vfio: Enable multiple groups per container > spapr_rtas_ddw: Workaround broken LE guests > target-ppc: kvm: make use of KVM_CREATE_SPAPR_TCE_64 > vfio: Enable in-kernel acceleration via VFIO KVM device > > hw/ppc/Makefile.objs | 3 + > hw/ppc/spapr.c | 5 + > hw/ppc/spapr_iommu.c | 23 ++- > hw/ppc/spapr_pci.c | 209 ++++++++++++++++++++------ > hw/ppc/spapr_pci_vfio.c | 108 +++++++++----- > hw/ppc/spapr_rtas.c | 29 ++++ > hw/ppc/spapr_rtas_ddw.c | 337 > ++++++++++++++++++++++++++++++++++++++++++ > hw/ppc/spapr_vio.c | 2 +- > hw/vfio/common.c | 167 ++++++++++++++++++--- > include/hw/pci-host/spapr.h | 38 ++++- > include/hw/ppc/spapr.h | 15 +- > include/hw/vfio/vfio-common.h | 3 +- > include/hw/vfio/vfio.h | 6 +- > target-ppc/kvm.c | 48 ++++-- > target-ppc/kvm_ppc.h | 10 +- > trace-events | 4 + > 16 files changed, 881 insertions(+), 126 deletions(-) > create mode 100644 hw/ppc/spapr_rtas_ddw.c > -- Alexey
