On Mon, 7 Mar 2022, at 10:33 PM, Alex Williamson wrote: > On Mon, 07 Mar 2022 22:13:01 +0000 > "Bronek Kozicki" <b...@incorrekt.com> wrote: > >> I know, this is such an old topic ... >> >> I have today upgraded my hypervisor from intel Ivy-Bridge to AMD Epyc >> Milan and, after making the necessary adjustments in vfio >> configuration to make my virtual machines work again, I found that >> there's immense amount of crackling (like, many times per second) in >> audio, both in Linux and in Windows guests. The audio is USB DAC >> connected through USB, and USB is on passed-through PCI (dedicated >> for guest). >> >> The host has two AMD 7413 CPUs, with lots of cores dedicated to >> guests: >> >> video=efifb:off iommu=pt amd_iommu=on add_efi_memmap >> nohz_full=6-35,54-83 rcu_nocbs=6-35,54-83 isolcpus=6-35,54-83 >> nvme.poll_queues=12 >> > > Getting interrupt bypass to work on AMD (AVIC) is rather more finicky > than the equivalent on Intel (APICv). Have a look at this pointer and > see if you can get it working: > > https://forum.level1techs.com/t/svm-avic-iommu-avic-improves-interrupt-performance-on-zen-2-etc-based-processors/155226 > > When working correctly, you should essentially stop seeing interrupt > counts increase in /proc/interrupts on the host for any vfio MSI > vectors. Thanks,
Hello Alex - thanks a lot for your help! I had no idea about AVIC. I went through all the steps in the guide you pointed me to, but it seems my kernels does not want to enable it. I think I need your help figuring out why. The effective kvm_amd parameters are: root@gdansk /sys/module/kvm_amd/parameters # grep -E '*' * avic:N dump_invalid_vmcb:N intercept_smi:Y nested:0 npt:Y nrips:1 pause_filter_count:3000 pause_filter_count_grow:2 pause_filter_count_max:65535 pause_filter_count_shrink:0 pause_filter_thresh:128 sev:N sev_es:N vgif:1 vls:1 Perhaps related, kvm effective parameters are: root@gdansk /sys/module/kvm/parameters # grep -E '*' * enable_vmware_backdoor:N flush_on_reuse:N force_emulation_prefix:N halt_poll_ns:200000 halt_poll_ns_grow:2 halt_poll_ns_grow_start:10000 halt_poll_ns_shrink:0 ignore_msrs:N kvmclock_periodic_sync:Y lapic_timer_advance_ns:-1 min_timer_period_us:200 mmio_caching:Y mmu_audit:N nx_huge_pages:N nx_huge_pages_recovery_ratio:60 pi_inject_timer:1 report_ignored_msrs:Y tdp_mmu:Y tsc_tolerance_ppm:250 vector_hashing:Y The parameters I wanted to apply are: root@gdansk /etc/modprobe.d (git)-[master] # cat kvm.conf options kvm ignore_msrs=0 options kvm_amd nested=0 avic=1 npt=1 I cannot see any errors in dmesg, but this is logged when I look for SVM root@gdansk /etc/modprobe.d (git)-[master] # dmesg -H -k --time-format=iso -L=always | grep SVM 2022-03-08T11:49:24,171231+00:00 SVM: kvm: Nested Paging enabled 2022-03-08T11:49:24,171257+00:00 SVM: Virtual VMLOAD VMSAVE supported 2022-03-08T11:49:24,171259+00:00 SVM: Virtual GIF supported My kernel is 5.15.26 vanilla ; my kernel config is: root@gdansk /etc (git)-[master] # zcat /proc/config.gz | grep -E "_KVM|_AMD" CONFIG_X86_AMD_PLATFORM_DEVICE=y CONFIG_KVM_GUEST=y CONFIG_CPU_SUP_AMD=y CONFIG_X86_MCE_AMD=y CONFIG_PERF_EVENTS_AMD_POWER=m CONFIG_PERF_EVENTS_AMD_UNCORE=y CONFIG_MICROCODE_AMD=y CONFIG_AMD_MEM_ENCRYPT=y # CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT is not set # CONFIG_AMD_NUMA is not set # CONFIG_X86_AMD_FREQ_SENSITIVITY is not set CONFIG_AMD_NB=y CONFIG_HAVE_KVM=y CONFIG_HAVE_KVM_IRQCHIP=y CONFIG_HAVE_KVM_IRQFD=y CONFIG_HAVE_KVM_IRQ_ROUTING=y CONFIG_HAVE_KVM_EVENTFD=y CONFIG_KVM_MMIO=y CONFIG_KVM_ASYNC_PF=y CONFIG_HAVE_KVM_MSI=y CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y CONFIG_KVM_VFIO=y CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y CONFIG_KVM_COMPAT=y CONFIG_HAVE_KVM_IRQ_BYPASS=y CONFIG_HAVE_KVM_NO_POLL=y CONFIG_KVM_XFER_TO_GUEST_WORK=y CONFIG_HAVE_KVM_PM_NOTIFIER=y CONFIG_KVM=m CONFIG_KVM_INTEL=m # CONFIG_X86_SGX_KVM is not set CONFIG_KVM_AMD=m CONFIG_KVM_AMD_SEV=y CONFIG_KVM_XEN=y CONFIG_KVM_MMU_AUDIT=y CONFIG_PATA_AMD=m CONFIG_NET_VENDOR_AMD=y CONFIG_AMD8111_ETH=m CONFIG_AMD_XGBE=m CONFIG_AMD_XGBE_DCB=y CONFIG_AMD_XGBE_HAVE_ECC=y CONFIG_AMD_PHY=m CONFIG_HW_RANDOM_AMD=m CONFIG_I2C_AMD756=m CONFIG_I2C_AMD756_S4882=m CONFIG_I2C_AMD8111=m CONFIG_I2C_AMD_MP2=m CONFIG_SPI_AMD=m CONFIG_PTP_1588_CLOCK_KVM=m CONFIG_PINCTRL_AMD=m CONFIG_GPIO_AMDPT=m CONFIG_GPIO_AMD_FCH=m CONFIG_GPIO_AMD8111=m CONFIG_AGP_AMD64=m CONFIG_DRM_AMDGPU=m CONFIG_DRM_AMDGPU_SI=y CONFIG_DRM_AMDGPU_CIK=y CONFIG_DRM_AMDGPU_USERPTR=y CONFIG_DRM_AMD_ACP=y CONFIG_DRM_AMD_DC=y CONFIG_DRM_AMD_DC_DCN=y CONFIG_DRM_AMD_DC_HDCP=y CONFIG_DRM_AMD_DC_SI=y # CONFIG_DRM_AMD_SECURE_DISPLAY is not set CONFIG_HSA_AMD=y CONFIG_HSA_AMD_SVM=y CONFIG_DRM_I915_GVT_KVMGT=m CONFIG_SND_SOC_AMD_ACP=m CONFIG_SND_SOC_AMD_CZ_DA7219MX98357_MACH=m CONFIG_SND_SOC_AMD_CZ_RT5645_MACH=m CONFIG_SND_SOC_AMD_ACP3x=m CONFIG_SND_SOC_AMD_RV_RT5682_MACH=m CONFIG_SND_SOC_AMD_RENOIR=m CONFIG_SND_SOC_AMD_RENOIR_MACH=m # CONFIG_SND_SOC_AMD_ACP5x is not set CONFIG_AMD_SFH_HID=m CONFIG_USB_AMD5536UDC=m CONFIG_EDAC_AMD64=m # CONFIG_AMD_PTDMA is not set CONFIG_AMD_PMC=m CONFIG_AMD_IOMMU=y CONFIG_AMD_IOMMU_V2=y CONFIG_NTB_AMD=m CONFIG_AMDTEE=m In case this is related I looked for APIC in dmesg, there is lots of it: 2022-03-08T11:49:19,000000+00:00 BIOS-e820: [mem 0x0000000074000000-0x0000000074046fff] ACPI NVS 2022-03-08T11:49:19,000000+00:00 BIOS-e820: [mem 0x00000000a823a000-0x00000000a8324fff] ACPI data 2022-03-08T11:49:19,000000+00:00 BIOS-e820: [mem 0x00000000a8325000-0x00000000a87a5fff] ACPI NVS 2022-03-08T11:49:19,000000+00:00 reserve setup_data: [mem 0x0000000074000000-0x0000000074046fff] ACPI NVS 2022-03-08T11:49:19,000000+00:00 reserve setup_data: [mem 0x00000000a823a000-0x00000000a8324fff] ACPI data 2022-03-08T11:49:19,000000+00:00 reserve setup_data: [mem 0x00000000a8325000-0x00000000a87a5fff] ACPI NVS 2022-03-08T11:49:19,000000+00:00 efi: ACPI=0xa878c000 ACPI 2.0=0xa878c014 SMBIOS=0xa9b23000 SMBIOS 3.0=0xa9b22000 MEMATTR=0x9f723018 ESRT=0xa1f5c998 2022-03-08T11:49:19,004800+00:00 ACPI: Early table checksum verification disabled 2022-03-08T11:49:19,004803+00:00 ACPI: RSDP 0x00000000A878C014 000024 (v02 SUPERM) 2022-03-08T11:49:19,004806+00:00 ACPI: XSDT 0x00000000A878B728 0000E4 (v01 SUPERM SMCI--MB 03242016 AMI 01000013) 2022-03-08T11:49:19,004810+00:00 ACPI: FACP 0x00000000A831E000 000114 (v06 SUPERM SMCI--MB 03242016 AMI 00010013) 2022-03-08T11:49:19,004814+00:00 ACPI: DSDT 0x00000000A830C000 01116E (v02 SUPERM SMCI--MB 03242016 INTL 20200717) 2022-03-08T11:49:19,004816+00:00 ACPI: FACS 0x00000000A8786000 000040 2022-03-08T11:49:19,004817+00:00 ACPI: SSDT 0x00000000A8324000 00092A (v02 SUPERM AmdTable 00000002 MSFT 04000000) 2022-03-08T11:49:19,004819+00:00 ACPI: SPMI 0x00000000A8323000 000041 (v05 SUPERM SMCI--MB 00000000 AMI. 00000000) 2022-03-08T11:49:19,004821+00:00 ACPI: SSDT 0x00000000A831F000 0033A1 (v02 SUPERM AMD AFC 00000001 INTL 20200717) 2022-03-08T11:49:19,004823+00:00 ACPI: FIDT 0x00000000A830B000 00009C (v01 SUPERM SMCI--MB 03242016 AMI 00010013) 2022-03-08T11:49:19,004825+00:00 ACPI: MCFG 0x00000000A830A000 00003C (v01 SUPERM SMCI--MB 03242016 MSFT 00010013) 2022-03-08T11:49:19,004826+00:00 ACPI: SSDT 0x00000000A8309000 000B0C (v02 SUPERM CPUSSDT 03242016 AMI 03242016) 2022-03-08T11:49:19,004828+00:00 ACPI: SSDT 0x00000000A8308000 000342 (v02 AMD CPMRAS 00000001 INTL 20200717) 2022-03-08T11:49:19,004830+00:00 ACPI: BERT 0x00000000A8307000 000030 (v01 AMD AmdTable 00000001 AMD 00000001) 2022-03-08T11:49:19,004831+00:00 ACPI: HPET 0x00000000A8305000 000038 (v01 SUPERM SMCI--MB 03242016 AMI 00000005) 2022-03-08T11:49:19,004833+00:00 ACPI: SPCR 0x00000000A8304000 000050 (v02 SUPERM SMCI--MB 03242016 AMI 00050016) 2022-03-08T11:49:19,004835+00:00 ACPI: IVRS 0x00000000A8303000 000370 (v02 SUPERM AmdTable 00000001 AMD 00000001) 2022-03-08T11:49:19,004836+00:00 ACPI: PCCT 0x00000000A8302000 00006E (v02 SUPERM AmdTable 00000001 AMD 00000001) 2022-03-08T11:49:19,004838+00:00 ACPI: SSDT 0x00000000A82EA000 017B09 (v02 SUPERM AmdTable 00000001 AMD 00000001) 2022-03-08T11:49:19,004840+00:00 ACPI: SRAT 0x00000000A82E9000 0006D0 (v03 SUPERM AmdTable 00000001 AMD 00000001) 2022-03-08T11:49:19,004841+00:00 ACPI: MSCT 0x00000000A82E8000 00004E (v01 SUPERM AmdTable 00000000 AMD 00000001) 2022-03-08T11:49:19,004843+00:00 ACPI: SLIT 0x00000000A82E7000 000030 (v01 SUPERM AmdTable 00000001 AMD 00000001) 2022-03-08T11:49:19,004845+00:00 ACPI: CRAT 0x00000000A82E1000 005720 (v01 SUPERM AmdTable 00000001 AMD 00000001) 2022-03-08T11:49:19,004846+00:00 ACPI: CDIT 0x00000000A82E0000 00002C (v01 SUPERM AmdTable 00000001 AMD 00000001) 2022-03-08T11:49:19,004848+00:00 ACPI: SSDT 0x00000000A82DE000 00182E (v02 AMD CPMCMN 00000001 INTL 20200717) 2022-03-08T11:49:19,004850+00:00 ACPI: WSMT 0x00000000A82DD000 000028 (v01 SUPERM SMCI--MB 03242016 AMI 00010013) 2022-03-08T11:49:19,004851+00:00 ACPI: APIC 0x00000000A82DC000 0008B2 (v04 SUPERM SMCI--MB 03242016 AMI 00010013) 2022-03-08T11:49:19,004853+00:00 ACPI: ERST 0x00000000A82DB000 000230 (v01 AMIER AMI.ERST 00000000 AMI. 00000000) 2022-03-08T11:49:19,004855+00:00 ACPI: HEST 0x00000000A8286000 054A74 (v01 AMD AmdTable 00000001 AMD 00000001) 2022-03-08T11:49:19,004856+00:00 ACPI: Reserving FACP table memory at [mem 0xa831e000-0xa831e113] 2022-03-08T11:49:19,004857+00:00 ACPI: Reserving DSDT table memory at [mem 0xa830c000-0xa831d16d] 2022-03-08T11:49:19,004858+00:00 ACPI: Reserving FACS table memory at [mem 0xa8786000-0xa878603f] 2022-03-08T11:49:19,004858+00:00 ACPI: Reserving SSDT table memory at [mem 0xa8324000-0xa8324929] 2022-03-08T11:49:19,004859+00:00 ACPI: Reserving SPMI table memory at [mem 0xa8323000-0xa8323040] 2022-03-08T11:49:19,004860+00:00 ACPI: Reserving SSDT table memory at [mem 0xa831f000-0xa83223a0] 2022-03-08T11:49:19,004860+00:00 ACPI: Reserving FIDT table memory at [mem 0xa830b000-0xa830b09b] 2022-03-08T11:49:19,004861+00:00 ACPI: Reserving MCFG table memory at [mem 0xa830a000-0xa830a03b] 2022-03-08T11:49:19,004862+00:00 ACPI: Reserving SSDT table memory at [mem 0xa8309000-0xa8309b0b] 2022-03-08T11:49:19,004862+00:00 ACPI: Reserving SSDT table memory at [mem 0xa8308000-0xa8308341] 2022-03-08T11:49:19,004863+00:00 ACPI: Reserving BERT table memory at [mem 0xa8307000-0xa830702f] 2022-03-08T11:49:19,004864+00:00 ACPI: Reserving HPET table memory at [mem 0xa8305000-0xa8305037] 2022-03-08T11:49:19,004864+00:00 ACPI: Reserving SPCR table memory at [mem 0xa8304000-0xa830404f] 2022-03-08T11:49:19,004865+00:00 ACPI: Reserving IVRS table memory at [mem 0xa8303000-0xa830336f] 2022-03-08T11:49:19,004865+00:00 ACPI: Reserving PCCT table memory at [mem 0xa8302000-0xa830206d] 2022-03-08T11:49:19,004866+00:00 ACPI: Reserving SSDT table memory at [mem 0xa82ea000-0xa8301b08] 2022-03-08T11:49:19,004867+00:00 ACPI: Reserving SRAT table memory at [mem 0xa82e9000-0xa82e96cf] 2022-03-08T11:49:19,004867+00:00 ACPI: Reserving MSCT table memory at [mem 0xa82e8000-0xa82e804d] 2022-03-08T11:49:19,004868+00:00 ACPI: Reserving SLIT table memory at [mem 0xa82e7000-0xa82e702f] 2022-03-08T11:49:19,004869+00:00 ACPI: Reserving CRAT table memory at [mem 0xa82e1000-0xa82e671f] 2022-03-08T11:49:19,004869+00:00 ACPI: Reserving CDIT table memory at [mem 0xa82e0000-0xa82e002b] 2022-03-08T11:49:19,004870+00:00 ACPI: Reserving SSDT table memory at [mem 0xa82de000-0xa82df82d] 2022-03-08T11:49:19,004871+00:00 ACPI: Reserving WSMT table memory at [mem 0xa82dd000-0xa82dd027] 2022-03-08T11:49:19,004871+00:00 ACPI: Reserving APIC table memory at [mem 0xa82dc000-0xa82dc8b1] 2022-03-08T11:49:19,004872+00:00 ACPI: Reserving ERST table memory at [mem 0xa82db000-0xa82db22f] 2022-03-08T11:49:19,004873+00:00 ACPI: Reserving HEST table memory at [mem 0xa8286000-0xa82daa73] 2022-03-08T11:49:19,004930+00:00 ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x0009ffff] 2022-03-08T11:49:19,004931+00:00 ACPI: SRAT: Node 0 PXM 0 [mem 0x000c0000-0xafffffff] 2022-03-08T11:49:19,004932+00:00 ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0x404fffffff] 2022-03-08T11:49:19,004932+00:00 ACPI: SRAT: Node 1 PXM 1 [mem 0x4050000000-0x804fffffff] 2022-03-08T11:49:19,515648+00:00 ACPI: PM-Timer IO Port: 0x808 2022-03-08T11:49:19,515661+00:00 ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) 2022-03-08T11:49:19,515716+00:00 ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) 2022-03-08T11:49:19,515718+00:00 ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level) 2022-03-08T11:49:19,515720+00:00 ACPI: Using ACPI (MADT) for SMP configuration information 2022-03-08T11:49:19,515721+00:00 ACPI: HPET id: 0x10228201 base: 0xfed00000 2022-03-08T11:49:19,515725+00:00 ACPI: SPCR: console: uart,io,0x3f8,115200 2022-03-08T11:49:20,163373+00:00 ACPI: Core revision 20210730 2022-03-08T11:49:20,745142+00:00 ACPI: PM: Registering ACPI NVS region [mem 0x74000000-0x74046fff] (290816 bytes) 2022-03-08T11:49:20,745142+00:00 ACPI: PM: Registering ACPI NVS region [mem 0xa8325000-0xa87a5fff] (4722688 bytes) 2022-03-08T11:49:20,745466+00:00 ACPI: bus type PCI registered 2022-03-08T11:49:20,745466+00:00 acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 2022-03-08T11:49:20,754215+00:00 ACPI: Added _OSI(Module Device) 2022-03-08T11:49:20,754215+00:00 ACPI: Added _OSI(Processor Device) 2022-03-08T11:49:20,754215+00:00 ACPI: Added _OSI(3.0 _SCP Extensions) 2022-03-08T11:49:20,754215+00:00 ACPI: Added _OSI(Processor Aggregator Device) 2022-03-08T11:49:20,754215+00:00 ACPI: Added _OSI(Linux-Dell-Video) 2022-03-08T11:49:20,754215+00:00 ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) 2022-03-08T11:49:20,754215+00:00 ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) 2022-03-08T11:49:20,773704+00:00 ACPI: 7 ACPI AML tables successfully acquired and loaded 2022-03-08T11:49:20,785121+00:00 ACPI: Interpreter enabled 2022-03-08T11:49:20,785131+00:00 ACPI: PM: (supports S0 S5) 2022-03-08T11:49:20,785132+00:00 ACPI: Using IOAPIC for interrupt routing 2022-03-08T11:49:20,785479+00:00 PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in ACPI motherboard resources 2022-03-08T11:49:20,863571+00:00 PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug 2022-03-08T11:49:20,865347+00:00 ACPI: Enabled 3 GPEs in block 00 to 1F 2022-03-08T11:49:20,866213+00:00 ACPI: PM: Power Resource [P0SA] 2022-03-08T11:49:20,866223+00:00 ACPI: PM: Power Resource [P3SA] . . . 2022-03-08T11:49:20,876335+00:00 ACPI: PM: Power Resource [P0SA] 2022-03-08T11:49:20,876344+00:00 ACPI: PM: Power Resource [P3SA] 2022-03-08T11:49:20,879548+00:00 ACPI: PCI Root Bridge [S0D0] (domain 0000 [bus 60-7f]) 2022-03-08T11:49:20,882867+00:00 ACPI: PCI Root Bridge [S0D1] (domain 0000 [bus 40-5f]) 2022-03-08T11:49:20,886662+00:00 ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f]) 2022-03-08T11:49:20,915589+00:00 ACPI: PCI Root Bridge [S0D2] (domain 0000 [bus 20-3f]) 2022-03-08T11:49:20,919320+00:00 ACPI: PCI Root Bridge [S1D0] (domain 0000 [bus e0-ff]) 2022-03-08T11:49:20,922529+00:00 ACPI: PCI Root Bridge [S1D1] (domain 0000 [bus c0-df]) 2022-03-08T11:49:20,926612+00:00 ACPI: PCI Root Bridge [S1D3] (domain 0000 [bus 80-9f]) 2022-03-08T11:49:20,929961+00:00 ACPI: PCI Root Bridge [S1D2] (domain 0000 [bus a0-bf]) 2022-03-08T11:49:20,955328+00:00 ACPI: PCI: Interrupt link LNKA configured for IRQ 11 2022-03-08T11:49:20,955373+00:00 ACPI: PCI: Interrupt link LNKB configured for IRQ 10 2022-03-08T11:49:20,955410+00:00 ACPI: PCI: Interrupt link LNKC configured for IRQ 11 2022-03-08T11:49:20,955456+00:00 ACPI: PCI: Interrupt link LNKD configured for IRQ 10 2022-03-08T11:49:20,955496+00:00 ACPI: PCI: Interrupt link LNKE configured for IRQ 5 2022-03-08T11:49:20,955531+00:00 ACPI: PCI: Interrupt link LNKF configured for IRQ 11 2022-03-08T11:49:20,955564+00:00 ACPI: PCI: Interrupt link LNKG configured for IRQ 10 2022-03-08T11:49:20,955598+00:00 ACPI: PCI: Interrupt link LNKH configured for IRQ 5 2022-03-08T11:49:20,956915+00:00 ACPI: bus type USB registered 2022-03-08T11:49:20,956915+00:00 PCI: Using ACPI for IRQ routing 2022-03-08T11:49:20,973685+00:00 pnp: PnP ACPI init 2022-03-08T11:49:20,977563+00:00 pnp: PnP ACPI: found 6 devices 2022-03-08T11:49:21,041524+00:00 ACPI: button: Power Button [PWRB] 2022-03-08T11:49:21,048614+00:00 ACPI: \_SB_.C000: Found 2 idle states . . . 2022-03-08T11:49:58,535255+00:00 ipmi_si IPI0001:00: ipmi_platform: probing via ACPI 2022-03-08T11:49:58,540252+00:00 ipmi_si dmi-ipmi-si.0: Removing SMBIOS-specified kcs state machine in favor of ACPI 2022-03-08T11:49:58,540253+00:00 ipmi_si: Adding ACPI-specified kcs state machine 2022-03-08T11:49:58,540352+00:00 ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0 In the machine BIOS most settings are on "Auto" B. -- Bronek Kozicki b...@incorrekt.com _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://listman.redhat.com/mailman/listinfo/vfio-users
