On POWER9 systems, the XICS-on-XIVE and XIVE KVM devices currently allocate a bunch of VPs in the XIVE HW to accomodate the highest VCPU id that may be possibly used in a VM. This limits the number of VMs that can run with an in-kernel interrupt controller to 63 per POWER9 chip, irrespectively of its number of HW threads, eg. up to 96 on a POWER9 Nimbus socket. This is an unfortunate waste of scarce HW resources since a typical VM doesn't need that much VPs to run.
This series exploits new attributes of the XICS-on-XIVE and XIVE KVM devices that allow userspace to tune the numbers of VPs it really needs. Patches 1 to 3 are preliminary work to teach the XICS and XIVE backends about the range of needed VCPU ids, according to the maximum number of VCPUs specified in the QEMU command line. Patch 5 and 6 do the actual work of configuring the KVM devices, based on new defines brought by a patch 4. RFC since the patches for KVM are still being discussed on the kvm-ppc list: https://patchwork.ozlabs.org/project/kvm-ppc/list/?series=132910 As a bonus, patch 7 allows the latest machine type to automatically set int KVM the guest core stride (VSMT) to be equal to the number of threads per core (-smp threads=N). This makes VCPU ids contiguous and allows to reduce the VP consumption even more. Both KVM and QEMU changes are available here: https://github.com/gkurz/linux/commits/xive-nr-servers-5.3 https://github.com/gkurz/qemu/commits/xive-nr-servers-for-4.2 -- Greg --- Greg Kurz (7): spapr, xics: Get number of servers with a XICSFabricClass method spapr, xive: Turn "nr-ends" property into "nr-servers" property spapr, xics, xive: Drop nr_servers argument in DT-related functions RFC linux-headers: Update against 5.3-rc2 spapr/xics: Configure number of servers in KVM spapr/xive: Configure number of servers in KVM spapr: Set VSMT to smp_threads by default hw/intc/spapr_xive.c | 24 ++++++++++++++++------ hw/intc/spapr_xive_kvm.c | 22 +++++++++++++++++++- hw/intc/xics.c | 7 +++++++ hw/intc/xics_kvm.c | 24 ++++++++++++++++++++-- hw/intc/xics_spapr.c | 6 +++--- hw/ppc/spapr.c | 18 ++++++++++++++--- hw/ppc/spapr_irq.c | 7 +++---- include/hw/ppc/spapr.h | 1 + include/hw/ppc/spapr_irq.h | 3 +-- include/hw/ppc/spapr_xive.h | 4 ++-- include/hw/ppc/xics.h | 2 ++ include/hw/ppc/xics_spapr.h | 3 +-- include/standard-headers/asm-x86/bootparam.h | 2 ++ include/standard-headers/asm-x86/kvm_para.h | 1 + include/standard-headers/linux/ethtool.h | 2 ++ include/standard-headers/linux/pci_regs.h | 4 ++++ include/standard-headers/linux/virtio_ids.h | 1 + include/standard-headers/linux/virtio_pmem.h | 6 +++--- linux-headers/asm-arm/kvm.h | 12 +++++++++++ linux-headers/asm-arm/unistd-common.h | 2 ++ linux-headers/asm-arm64/kvm.h | 17 ++++++++++++++++ linux-headers/asm-generic/mman-common.h | 15 ++++++++------ linux-headers/asm-generic/mman.h | 10 ++++----- linux-headers/asm-generic/unistd.h | 8 +++++++ linux-headers/asm-mips/unistd_n32.h | 1 + linux-headers/asm-mips/unistd_n64.h | 1 + linux-headers/asm-mips/unistd_o32.h | 1 + linux-headers/asm-powerpc/kvm.h | 3 +++ linux-headers/asm-powerpc/mman.h | 6 +----- linux-headers/asm-powerpc/unistd_32.h | 2 ++ linux-headers/asm-powerpc/unistd_64.h | 2 ++ linux-headers/asm-s390/unistd_32.h | 2 ++ linux-headers/asm-s390/unistd_64.h | 2 ++ linux-headers/asm-x86/kvm.h | 28 ++++++++++++++++++++------ linux-headers/asm-x86/unistd_32.h | 2 ++ linux-headers/asm-x86/unistd_64.h | 2 ++ linux-headers/asm-x86/unistd_x32.h | 2 ++ linux-headers/linux/kvm.h | 7 +++++-- linux-headers/linux/psp-sev.h | 5 +---- 39 files changed, 206 insertions(+), 61 deletions(-)
