Re: [Qemu-devel] [PATCH v6 00/16] KVM platform device passthrough
On 09/16/2014 11:23 PM, Alex Williamson wrote: On Tue, 2014-09-16 at 14:51 -0600, Alex Williamson wrote: On Tue, 2014-09-16 at 00:01 +0200, Eric Auger wrote: On 09/12/2014 01:05 AM, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:51:14PM -0600, Alex Williamson wrote: On Thu, 2014-09-11 at 15:23 -0700, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:14:09PM -0600, Alex Williamson wrote: On Tue, 2014-09-09 at 08:31 +0100, Eric Auger wrote: This RFC series aims at enabling KVM platform device passthrough. It implements a VFIO platform device, derived from VFIO PCI device. The VFIO platform device uses the host VFIO platform driver which must be bound to the assigned device prior to the QEMU system start. - the guest can directly access the device register space - assigned device IRQs are transparently routed to the guest by QEMU/KVM (3 methods currently are supported: user-level eventfd handling, irqfd, forwarded IRQs) - iommu is transparently programmed to prevent the device from accessing physical pages outside of the guest address space This patch series is made of the following patch files: 1-7) Modifications to PCI code to prepare for VFIO platform device 8) split of PCI specific code and generic code (move) 9-11) creation of the VFIO calxeda xgmac platform device, without irqfd support (MMIO direct access and IRQ assignment). 12) fake injection test modality (to test multiple IRQ) 13) addition of irqfd/virqfd support 14-16) forwarded IRQ Dependency List: QEMU dependencies: [1] [PATCH v2 0/9] Dynamic sysbus device allocation support, Alex Graf http://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00047.html [2] [RFC v3] machvirt dynamic sysbus device instantiation, Eric Auger [3] [PATCH v2 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html [4] [RFC] vfio: migration to trace points, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00569.html Kernel Dependencies: [5] [RFC Patch v6 0/20] VFIO support for platform devices, Antonios Motakis https://www.mail-archive.com/kvm@vger.kernel.org/msg103247.html [6] [PATCH v3] ARM: KVM: add irqfd support, Eric Auger https://lkml.org/lkml/2014/9/1/141 [7] arm/arm64: KVM: Various VGIC cleanups and improvements, Christoffer Dall http://comments.gmane.org/gmane.linux.ports.arm.kernel/340430 [8] [RFC v2 0/9] KVM-VFIO IRQ forward control, Eric Auger https://lkml.org/lkml/2014/9/1/344 [9] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, Marc Zyngier http://lwn.net/Articles/603514/ kernel pieces can be found at: http://git.linaro.org/people/eric.auger/linux.git (branch 3.17rc3_irqfd_forward_integ_v2) QEMU pieces can be found at: http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v6) The patch series was tested on Calxeda Midway (ARMv7) where one xgmac is assigned to KVM host while the second one is assigned to the guest. Reworked PCI device is not tested. Wiki for Calxeda Midway setup: https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway History: v5-v6: - rebase on 2.1rc5 PCI code - forwarded IRQ first integraton Why? Are there acceleration paths that you're concerned cannot be implemented or we do not already have a proof of concept for? The base kernel patch series you depend on is 3 months old yet this series continues to grow and add new dependencies. Please let's prioritize getting something upstream instead of adding more blockers to prevent that. Thanks, I'm not exactly sure what this changelog line was referring to (depending on Marc's forwarding IRQ patches?), but just want to add that there are a number of dependencies for the GIC that need to go in as well (should happen within a few weeks), but I think it's unlikely that the IRQ forwarding stuff goes in for v3.18 at this point. It may make sense as you suggest to keep that part out of this patch set and something merged sooner as opposed to later, but I'm too jet-lagged to completely understand if that's going to be a horrible mess. The point is that we're on v6 of a patch series and its first non-RFC posting and we're rolling in a first pass at a QEMU implementation that depends on a contested kernel RFC, which depends on another stagnant kernel RFC. I'm fine with working on it in parallel, but give me some light at the end of the tunnel as a reviewer and maintainer that this code isn't going to live indefinitely on the mailing list. Do we really need those GIC patches do be able to have non-KVM accelerated VFIO platform device assignment? We certainly don't need IRQ forwarding. Thanks, Hi Alex, Sorry for the delay, I was travelling. I understand your impatience. I personally would be happy if we could envision upstreaming this patch in several
Re: [Qemu-devel] [PATCH v6 00/16] KVM platform device passthrough
On Tue, 2014-09-16 at 00:01 +0200, Eric Auger wrote: On 09/12/2014 01:05 AM, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:51:14PM -0600, Alex Williamson wrote: On Thu, 2014-09-11 at 15:23 -0700, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:14:09PM -0600, Alex Williamson wrote: On Tue, 2014-09-09 at 08:31 +0100, Eric Auger wrote: This RFC series aims at enabling KVM platform device passthrough. It implements a VFIO platform device, derived from VFIO PCI device. The VFIO platform device uses the host VFIO platform driver which must be bound to the assigned device prior to the QEMU system start. - the guest can directly access the device register space - assigned device IRQs are transparently routed to the guest by QEMU/KVM (3 methods currently are supported: user-level eventfd handling, irqfd, forwarded IRQs) - iommu is transparently programmed to prevent the device from accessing physical pages outside of the guest address space This patch series is made of the following patch files: 1-7) Modifications to PCI code to prepare for VFIO platform device 8) split of PCI specific code and generic code (move) 9-11) creation of the VFIO calxeda xgmac platform device, without irqfd support (MMIO direct access and IRQ assignment). 12) fake injection test modality (to test multiple IRQ) 13) addition of irqfd/virqfd support 14-16) forwarded IRQ Dependency List: QEMU dependencies: [1] [PATCH v2 0/9] Dynamic sysbus device allocation support, Alex Graf http://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00047.html [2] [RFC v3] machvirt dynamic sysbus device instantiation, Eric Auger [3] [PATCH v2 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html [4] [RFC] vfio: migration to trace points, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00569.html Kernel Dependencies: [5] [RFC Patch v6 0/20] VFIO support for platform devices, Antonios Motakis https://www.mail-archive.com/kvm@vger.kernel.org/msg103247.html [6] [PATCH v3] ARM: KVM: add irqfd support, Eric Auger https://lkml.org/lkml/2014/9/1/141 [7] arm/arm64: KVM: Various VGIC cleanups and improvements, Christoffer Dall http://comments.gmane.org/gmane.linux.ports.arm.kernel/340430 [8] [RFC v2 0/9] KVM-VFIO IRQ forward control, Eric Auger https://lkml.org/lkml/2014/9/1/344 [9] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, Marc Zyngier http://lwn.net/Articles/603514/ kernel pieces can be found at: http://git.linaro.org/people/eric.auger/linux.git (branch 3.17rc3_irqfd_forward_integ_v2) QEMU pieces can be found at: http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v6) The patch series was tested on Calxeda Midway (ARMv7) where one xgmac is assigned to KVM host while the second one is assigned to the guest. Reworked PCI device is not tested. Wiki for Calxeda Midway setup: https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway History: v5-v6: - rebase on 2.1rc5 PCI code - forwarded IRQ first integraton Why? Are there acceleration paths that you're concerned cannot be implemented or we do not already have a proof of concept for? The base kernel patch series you depend on is 3 months old yet this series continues to grow and add new dependencies. Please let's prioritize getting something upstream instead of adding more blockers to prevent that. Thanks, I'm not exactly sure what this changelog line was referring to (depending on Marc's forwarding IRQ patches?), but just want to add that there are a number of dependencies for the GIC that need to go in as well (should happen within a few weeks), but I think it's unlikely that the IRQ forwarding stuff goes in for v3.18 at this point. It may make sense as you suggest to keep that part out of this patch set and something merged sooner as opposed to later, but I'm too jet-lagged to completely understand if that's going to be a horrible mess. The point is that we're on v6 of a patch series and its first non-RFC posting and we're rolling in a first pass at a QEMU implementation that depends on a contested kernel RFC, which depends on another stagnant kernel RFC. I'm fine with working on it in parallel, but give me some light at the end of the tunnel as a reviewer and maintainer that this code isn't going to live indefinitely on the mailing list. Do we really need those GIC patches do be able to have non-KVM accelerated VFIO platform device assignment? We certainly don't need IRQ forwarding. Thanks, Hi Alex, Sorry for the delay, I was travelling. I understand your impatience. I personally would be happy if we could envision upstreaming this patch in several
Re: [Qemu-devel] [PATCH v6 00/16] KVM platform device passthrough
On 09/16/2014 10:51 PM, Alex Williamson wrote: On Tue, 2014-09-16 at 00:01 +0200, Eric Auger wrote: On 09/12/2014 01:05 AM, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:51:14PM -0600, Alex Williamson wrote: On Thu, 2014-09-11 at 15:23 -0700, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:14:09PM -0600, Alex Williamson wrote: On Tue, 2014-09-09 at 08:31 +0100, Eric Auger wrote: This RFC series aims at enabling KVM platform device passthrough. It implements a VFIO platform device, derived from VFIO PCI device. The VFIO platform device uses the host VFIO platform driver which must be bound to the assigned device prior to the QEMU system start. - the guest can directly access the device register space - assigned device IRQs are transparently routed to the guest by QEMU/KVM (3 methods currently are supported: user-level eventfd handling, irqfd, forwarded IRQs) - iommu is transparently programmed to prevent the device from accessing physical pages outside of the guest address space This patch series is made of the following patch files: 1-7) Modifications to PCI code to prepare for VFIO platform device 8) split of PCI specific code and generic code (move) 9-11) creation of the VFIO calxeda xgmac platform device, without irqfd support (MMIO direct access and IRQ assignment). 12) fake injection test modality (to test multiple IRQ) 13) addition of irqfd/virqfd support 14-16) forwarded IRQ Dependency List: QEMU dependencies: [1] [PATCH v2 0/9] Dynamic sysbus device allocation support, Alex Graf http://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00047.html [2] [RFC v3] machvirt dynamic sysbus device instantiation, Eric Auger [3] [PATCH v2 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html [4] [RFC] vfio: migration to trace points, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00569.html Kernel Dependencies: [5] [RFC Patch v6 0/20] VFIO support for platform devices, Antonios Motakis https://www.mail-archive.com/kvm@vger.kernel.org/msg103247.html [6] [PATCH v3] ARM: KVM: add irqfd support, Eric Auger https://lkml.org/lkml/2014/9/1/141 [7] arm/arm64: KVM: Various VGIC cleanups and improvements, Christoffer Dall http://comments.gmane.org/gmane.linux.ports.arm.kernel/340430 [8] [RFC v2 0/9] KVM-VFIO IRQ forward control, Eric Auger https://lkml.org/lkml/2014/9/1/344 [9] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, Marc Zyngier http://lwn.net/Articles/603514/ kernel pieces can be found at: http://git.linaro.org/people/eric.auger/linux.git (branch 3.17rc3_irqfd_forward_integ_v2) QEMU pieces can be found at: http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v6) The patch series was tested on Calxeda Midway (ARMv7) where one xgmac is assigned to KVM host while the second one is assigned to the guest. Reworked PCI device is not tested. Wiki for Calxeda Midway setup: https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway History: v5-v6: - rebase on 2.1rc5 PCI code - forwarded IRQ first integraton Why? Are there acceleration paths that you're concerned cannot be implemented or we do not already have a proof of concept for? The base kernel patch series you depend on is 3 months old yet this series continues to grow and add new dependencies. Please let's prioritize getting something upstream instead of adding more blockers to prevent that. Thanks, I'm not exactly sure what this changelog line was referring to (depending on Marc's forwarding IRQ patches?), but just want to add that there are a number of dependencies for the GIC that need to go in as well (should happen within a few weeks), but I think it's unlikely that the IRQ forwarding stuff goes in for v3.18 at this point. It may make sense as you suggest to keep that part out of this patch set and something merged sooner as opposed to later, but I'm too jet-lagged to completely understand if that's going to be a horrible mess. The point is that we're on v6 of a patch series and its first non-RFC posting and we're rolling in a first pass at a QEMU implementation that depends on a contested kernel RFC, which depends on another stagnant kernel RFC. I'm fine with working on it in parallel, but give me some light at the end of the tunnel as a reviewer and maintainer that this code isn't going to live indefinitely on the mailing list. Do we really need those GIC patches do be able to have non-KVM accelerated VFIO platform device assignment? We certainly don't need IRQ forwarding. Thanks, Hi Alex, Sorry for the delay, I was travelling. I understand your impatience. I personally would be happy if we could envision upstreaming this patch in several steps. Let me know if it makes sense. STEP I: integrate 1
Re: [Qemu-devel] [PATCH v6 00/16] KVM platform device passthrough
On Tue, 2014-09-16 at 14:51 -0600, Alex Williamson wrote: On Tue, 2014-09-16 at 00:01 +0200, Eric Auger wrote: On 09/12/2014 01:05 AM, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:51:14PM -0600, Alex Williamson wrote: On Thu, 2014-09-11 at 15:23 -0700, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:14:09PM -0600, Alex Williamson wrote: On Tue, 2014-09-09 at 08:31 +0100, Eric Auger wrote: This RFC series aims at enabling KVM platform device passthrough. It implements a VFIO platform device, derived from VFIO PCI device. The VFIO platform device uses the host VFIO platform driver which must be bound to the assigned device prior to the QEMU system start. - the guest can directly access the device register space - assigned device IRQs are transparently routed to the guest by QEMU/KVM (3 methods currently are supported: user-level eventfd handling, irqfd, forwarded IRQs) - iommu is transparently programmed to prevent the device from accessing physical pages outside of the guest address space This patch series is made of the following patch files: 1-7) Modifications to PCI code to prepare for VFIO platform device 8) split of PCI specific code and generic code (move) 9-11) creation of the VFIO calxeda xgmac platform device, without irqfd support (MMIO direct access and IRQ assignment). 12) fake injection test modality (to test multiple IRQ) 13) addition of irqfd/virqfd support 14-16) forwarded IRQ Dependency List: QEMU dependencies: [1] [PATCH v2 0/9] Dynamic sysbus device allocation support, Alex Graf http://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00047.html [2] [RFC v3] machvirt dynamic sysbus device instantiation, Eric Auger [3] [PATCH v2 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html [4] [RFC] vfio: migration to trace points, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00569.html Kernel Dependencies: [5] [RFC Patch v6 0/20] VFIO support for platform devices, Antonios Motakis https://www.mail-archive.com/kvm@vger.kernel.org/msg103247.html [6] [PATCH v3] ARM: KVM: add irqfd support, Eric Auger https://lkml.org/lkml/2014/9/1/141 [7] arm/arm64: KVM: Various VGIC cleanups and improvements, Christoffer Dall http://comments.gmane.org/gmane.linux.ports.arm.kernel/340430 [8] [RFC v2 0/9] KVM-VFIO IRQ forward control, Eric Auger https://lkml.org/lkml/2014/9/1/344 [9] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, Marc Zyngier http://lwn.net/Articles/603514/ kernel pieces can be found at: http://git.linaro.org/people/eric.auger/linux.git (branch 3.17rc3_irqfd_forward_integ_v2) QEMU pieces can be found at: http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v6) The patch series was tested on Calxeda Midway (ARMv7) where one xgmac is assigned to KVM host while the second one is assigned to the guest. Reworked PCI device is not tested. Wiki for Calxeda Midway setup: https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway History: v5-v6: - rebase on 2.1rc5 PCI code - forwarded IRQ first integraton Why? Are there acceleration paths that you're concerned cannot be implemented or we do not already have a proof of concept for? The base kernel patch series you depend on is 3 months old yet this series continues to grow and add new dependencies. Please let's prioritize getting something upstream instead of adding more blockers to prevent that. Thanks, I'm not exactly sure what this changelog line was referring to (depending on Marc's forwarding IRQ patches?), but just want to add that there are a number of dependencies for the GIC that need to go in as well (should happen within a few weeks), but I think it's unlikely that the IRQ forwarding stuff goes in for v3.18 at this point. It may make sense as you suggest to keep that part out of this patch set and something merged sooner as opposed to later, but I'm too jet-lagged to completely understand if that's going to be a horrible mess. The point is that we're on v6 of a patch series and its first non-RFC posting and we're rolling in a first pass at a QEMU implementation that depends on a contested kernel RFC, which depends on another stagnant kernel RFC. I'm fine with working on it in parallel, but give me some light at the end of the tunnel as a reviewer and maintainer that this code isn't going to live indefinitely on the mailing list. Do we really need those GIC patches do be able to have non-KVM accelerated VFIO platform device assignment? We certainly don't need IRQ forwarding. Thanks,
Re: [Qemu-devel] [PATCH v6 00/16] KVM platform device passthrough
On 09/12/2014 01:05 AM, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:51:14PM -0600, Alex Williamson wrote: On Thu, 2014-09-11 at 15:23 -0700, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:14:09PM -0600, Alex Williamson wrote: On Tue, 2014-09-09 at 08:31 +0100, Eric Auger wrote: This RFC series aims at enabling KVM platform device passthrough. It implements a VFIO platform device, derived from VFIO PCI device. The VFIO platform device uses the host VFIO platform driver which must be bound to the assigned device prior to the QEMU system start. - the guest can directly access the device register space - assigned device IRQs are transparently routed to the guest by QEMU/KVM (3 methods currently are supported: user-level eventfd handling, irqfd, forwarded IRQs) - iommu is transparently programmed to prevent the device from accessing physical pages outside of the guest address space This patch series is made of the following patch files: 1-7) Modifications to PCI code to prepare for VFIO platform device 8) split of PCI specific code and generic code (move) 9-11) creation of the VFIO calxeda xgmac platform device, without irqfd support (MMIO direct access and IRQ assignment). 12) fake injection test modality (to test multiple IRQ) 13) addition of irqfd/virqfd support 14-16) forwarded IRQ Dependency List: QEMU dependencies: [1] [PATCH v2 0/9] Dynamic sysbus device allocation support, Alex Graf http://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00047.html [2] [RFC v3] machvirt dynamic sysbus device instantiation, Eric Auger [3] [PATCH v2 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html [4] [RFC] vfio: migration to trace points, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00569.html Kernel Dependencies: [5] [RFC Patch v6 0/20] VFIO support for platform devices, Antonios Motakis https://www.mail-archive.com/kvm@vger.kernel.org/msg103247.html [6] [PATCH v3] ARM: KVM: add irqfd support, Eric Auger https://lkml.org/lkml/2014/9/1/141 [7] arm/arm64: KVM: Various VGIC cleanups and improvements, Christoffer Dall http://comments.gmane.org/gmane.linux.ports.arm.kernel/340430 [8] [RFC v2 0/9] KVM-VFIO IRQ forward control, Eric Auger https://lkml.org/lkml/2014/9/1/344 [9] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, Marc Zyngier http://lwn.net/Articles/603514/ kernel pieces can be found at: http://git.linaro.org/people/eric.auger/linux.git (branch 3.17rc3_irqfd_forward_integ_v2) QEMU pieces can be found at: http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v6) The patch series was tested on Calxeda Midway (ARMv7) where one xgmac is assigned to KVM host while the second one is assigned to the guest. Reworked PCI device is not tested. Wiki for Calxeda Midway setup: https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway History: v5-v6: - rebase on 2.1rc5 PCI code - forwarded IRQ first integraton Why? Are there acceleration paths that you're concerned cannot be implemented or we do not already have a proof of concept for? The base kernel patch series you depend on is 3 months old yet this series continues to grow and add new dependencies. Please let's prioritize getting something upstream instead of adding more blockers to prevent that. Thanks, I'm not exactly sure what this changelog line was referring to (depending on Marc's forwarding IRQ patches?), but just want to add that there are a number of dependencies for the GIC that need to go in as well (should happen within a few weeks), but I think it's unlikely that the IRQ forwarding stuff goes in for v3.18 at this point. It may make sense as you suggest to keep that part out of this patch set and something merged sooner as opposed to later, but I'm too jet-lagged to completely understand if that's going to be a horrible mess. The point is that we're on v6 of a patch series and its first non-RFC posting and we're rolling in a first pass at a QEMU implementation that depends on a contested kernel RFC, which depends on another stagnant kernel RFC. I'm fine with working on it in parallel, but give me some light at the end of the tunnel as a reviewer and maintainer that this code isn't going to live indefinitely on the mailing list. Do we really need those GIC patches do be able to have non-KVM accelerated VFIO platform device assignment? We certainly don't need IRQ forwarding. Thanks, Hi Alex, Sorry for the delay, I was travelling. I understand your impatience. I personally would be happy if we could envision upstreaming this patch in several steps. Let me know if it makes sense. STEP I: integrate 1 - 11: leads to have a non-KVM accelerated VFIO QEMU device. 12 can be part of it too but since it is a test
Re: [Qemu-devel] [PATCH v6 00/16] KVM platform device passthrough
On Tue, 2014-09-09 at 08:31 +0100, Eric Auger wrote: This RFC series aims at enabling KVM platform device passthrough. It implements a VFIO platform device, derived from VFIO PCI device. The VFIO platform device uses the host VFIO platform driver which must be bound to the assigned device prior to the QEMU system start. - the guest can directly access the device register space - assigned device IRQs are transparently routed to the guest by QEMU/KVM (3 methods currently are supported: user-level eventfd handling, irqfd, forwarded IRQs) - iommu is transparently programmed to prevent the device from accessing physical pages outside of the guest address space This patch series is made of the following patch files: 1-7) Modifications to PCI code to prepare for VFIO platform device 8) split of PCI specific code and generic code (move) 9-11) creation of the VFIO calxeda xgmac platform device, without irqfd support (MMIO direct access and IRQ assignment). 12) fake injection test modality (to test multiple IRQ) 13) addition of irqfd/virqfd support 14-16) forwarded IRQ Dependency List: QEMU dependencies: [1] [PATCH v2 0/9] Dynamic sysbus device allocation support, Alex Graf http://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00047.html [2] [RFC v3] machvirt dynamic sysbus device instantiation, Eric Auger [3] [PATCH v2 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html [4] [RFC] vfio: migration to trace points, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00569.html Kernel Dependencies: [5] [RFC Patch v6 0/20] VFIO support for platform devices, Antonios Motakis https://www.mail-archive.com/kvm@vger.kernel.org/msg103247.html [6] [PATCH v3] ARM: KVM: add irqfd support, Eric Auger https://lkml.org/lkml/2014/9/1/141 [7] arm/arm64: KVM: Various VGIC cleanups and improvements, Christoffer Dall http://comments.gmane.org/gmane.linux.ports.arm.kernel/340430 [8] [RFC v2 0/9] KVM-VFIO IRQ forward control, Eric Auger https://lkml.org/lkml/2014/9/1/344 [9] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, Marc Zyngier http://lwn.net/Articles/603514/ kernel pieces can be found at: http://git.linaro.org/people/eric.auger/linux.git (branch 3.17rc3_irqfd_forward_integ_v2) QEMU pieces can be found at: http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v6) The patch series was tested on Calxeda Midway (ARMv7) where one xgmac is assigned to KVM host while the second one is assigned to the guest. Reworked PCI device is not tested. Wiki for Calxeda Midway setup: https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway History: v5-v6: - rebase on 2.1rc5 PCI code - forwarded IRQ first integraton Why? Are there acceleration paths that you're concerned cannot be implemented or we do not already have a proof of concept for? The base kernel patch series you depend on is 3 months old yet this series continues to grow and add new dependencies. Please let's prioritize getting something upstream instead of adding more blockers to prevent that. Thanks, Alex - vfio_device property renamed into host property - split IRQ setup in different functions that match the 3 supported injection techniques (user handled eventfd, irqfd, forwarded IRQ): removes dynamic switch between injection methods - introduce fake interrupts as a test modality: x makes possible to test multiple IRQ user-side handling. x this is a test feature only: enable to trigger a fd as if the real physical IRQ hit. No virtual IRQ is injected into the guest but handling is simulated so that the state machine can be tested - user handled eventfd: x add mutex to protect IRQ state list manipulation, x correct misleading comment in vfio_intp_interrupt. x Fix bugs using fake interrupt modality - irqfd no more advertised in this patchset (handled in [3]) - VFIOPlatformDeviceClass becomes abstract and Calxeda xgmac device and class is re-introduced (as per v4) - all DPRINTF removed in platform and replaced by trace-points - corrects compilation with configure --disable-kvm - simplifies the split for vfio_get_device and introduce a unique specialized function named vfio_populate_device - group_list renamed into vfio_group_list - hw/arm/dyn_sysbus_devtree.c currently only support vfio-calxeda-xgmac instantiation. Needs to be specialized for other VFIO devices - fix 2 bugs in dyn_sysbus_devtree(reg_attr index and compat) v4-v5: - rebase on v2.1.0 PCI code - take into account Alex Williamson comments on PCI code rework - trace updates in vfio_region_write/read - remove fd from VFIORegion - get/put ckeanup - bug fix: bar region's vbasedev field duly initialization - misc cleanups in platform device - device
Re: [Qemu-devel] [PATCH v6 00/16] KVM platform device passthrough
On Thu, Sep 11, 2014 at 04:14:09PM -0600, Alex Williamson wrote: On Tue, 2014-09-09 at 08:31 +0100, Eric Auger wrote: This RFC series aims at enabling KVM platform device passthrough. It implements a VFIO platform device, derived from VFIO PCI device. The VFIO platform device uses the host VFIO platform driver which must be bound to the assigned device prior to the QEMU system start. - the guest can directly access the device register space - assigned device IRQs are transparently routed to the guest by QEMU/KVM (3 methods currently are supported: user-level eventfd handling, irqfd, forwarded IRQs) - iommu is transparently programmed to prevent the device from accessing physical pages outside of the guest address space This patch series is made of the following patch files: 1-7) Modifications to PCI code to prepare for VFIO platform device 8) split of PCI specific code and generic code (move) 9-11) creation of the VFIO calxeda xgmac platform device, without irqfd support (MMIO direct access and IRQ assignment). 12) fake injection test modality (to test multiple IRQ) 13) addition of irqfd/virqfd support 14-16) forwarded IRQ Dependency List: QEMU dependencies: [1] [PATCH v2 0/9] Dynamic sysbus device allocation support, Alex Graf http://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00047.html [2] [RFC v3] machvirt dynamic sysbus device instantiation, Eric Auger [3] [PATCH v2 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html [4] [RFC] vfio: migration to trace points, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00569.html Kernel Dependencies: [5] [RFC Patch v6 0/20] VFIO support for platform devices, Antonios Motakis https://www.mail-archive.com/kvm@vger.kernel.org/msg103247.html [6] [PATCH v3] ARM: KVM: add irqfd support, Eric Auger https://lkml.org/lkml/2014/9/1/141 [7] arm/arm64: KVM: Various VGIC cleanups and improvements, Christoffer Dall http://comments.gmane.org/gmane.linux.ports.arm.kernel/340430 [8] [RFC v2 0/9] KVM-VFIO IRQ forward control, Eric Auger https://lkml.org/lkml/2014/9/1/344 [9] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, Marc Zyngier http://lwn.net/Articles/603514/ kernel pieces can be found at: http://git.linaro.org/people/eric.auger/linux.git (branch 3.17rc3_irqfd_forward_integ_v2) QEMU pieces can be found at: http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v6) The patch series was tested on Calxeda Midway (ARMv7) where one xgmac is assigned to KVM host while the second one is assigned to the guest. Reworked PCI device is not tested. Wiki for Calxeda Midway setup: https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway History: v5-v6: - rebase on 2.1rc5 PCI code - forwarded IRQ first integraton Why? Are there acceleration paths that you're concerned cannot be implemented or we do not already have a proof of concept for? The base kernel patch series you depend on is 3 months old yet this series continues to grow and add new dependencies. Please let's prioritize getting something upstream instead of adding more blockers to prevent that. Thanks, I'm not exactly sure what this changelog line was referring to (depending on Marc's forwarding IRQ patches?), but just want to add that there are a number of dependencies for the GIC that need to go in as well (should happen within a few weeks), but I think it's unlikely that the IRQ forwarding stuff goes in for v3.18 at this point. It may make sense as you suggest to keep that part out of this patch set and something merged sooner as opposed to later, but I'm too jet-lagged to completely understand if that's going to be a horrible mess. Thanks, -Christoffer
Re: [Qemu-devel] [PATCH v6 00/16] KVM platform device passthrough
On Thu, 2014-09-11 at 15:23 -0700, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:14:09PM -0600, Alex Williamson wrote: On Tue, 2014-09-09 at 08:31 +0100, Eric Auger wrote: This RFC series aims at enabling KVM platform device passthrough. It implements a VFIO platform device, derived from VFIO PCI device. The VFIO platform device uses the host VFIO platform driver which must be bound to the assigned device prior to the QEMU system start. - the guest can directly access the device register space - assigned device IRQs are transparently routed to the guest by QEMU/KVM (3 methods currently are supported: user-level eventfd handling, irqfd, forwarded IRQs) - iommu is transparently programmed to prevent the device from accessing physical pages outside of the guest address space This patch series is made of the following patch files: 1-7) Modifications to PCI code to prepare for VFIO platform device 8) split of PCI specific code and generic code (move) 9-11) creation of the VFIO calxeda xgmac platform device, without irqfd support (MMIO direct access and IRQ assignment). 12) fake injection test modality (to test multiple IRQ) 13) addition of irqfd/virqfd support 14-16) forwarded IRQ Dependency List: QEMU dependencies: [1] [PATCH v2 0/9] Dynamic sysbus device allocation support, Alex Graf http://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00047.html [2] [RFC v3] machvirt dynamic sysbus device instantiation, Eric Auger [3] [PATCH v2 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html [4] [RFC] vfio: migration to trace points, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00569.html Kernel Dependencies: [5] [RFC Patch v6 0/20] VFIO support for platform devices, Antonios Motakis https://www.mail-archive.com/kvm@vger.kernel.org/msg103247.html [6] [PATCH v3] ARM: KVM: add irqfd support, Eric Auger https://lkml.org/lkml/2014/9/1/141 [7] arm/arm64: KVM: Various VGIC cleanups and improvements, Christoffer Dall http://comments.gmane.org/gmane.linux.ports.arm.kernel/340430 [8] [RFC v2 0/9] KVM-VFIO IRQ forward control, Eric Auger https://lkml.org/lkml/2014/9/1/344 [9] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, Marc Zyngier http://lwn.net/Articles/603514/ kernel pieces can be found at: http://git.linaro.org/people/eric.auger/linux.git (branch 3.17rc3_irqfd_forward_integ_v2) QEMU pieces can be found at: http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v6) The patch series was tested on Calxeda Midway (ARMv7) where one xgmac is assigned to KVM host while the second one is assigned to the guest. Reworked PCI device is not tested. Wiki for Calxeda Midway setup: https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway History: v5-v6: - rebase on 2.1rc5 PCI code - forwarded IRQ first integraton Why? Are there acceleration paths that you're concerned cannot be implemented or we do not already have a proof of concept for? The base kernel patch series you depend on is 3 months old yet this series continues to grow and add new dependencies. Please let's prioritize getting something upstream instead of adding more blockers to prevent that. Thanks, I'm not exactly sure what this changelog line was referring to (depending on Marc's forwarding IRQ patches?), but just want to add that there are a number of dependencies for the GIC that need to go in as well (should happen within a few weeks), but I think it's unlikely that the IRQ forwarding stuff goes in for v3.18 at this point. It may make sense as you suggest to keep that part out of this patch set and something merged sooner as opposed to later, but I'm too jet-lagged to completely understand if that's going to be a horrible mess. The point is that we're on v6 of a patch series and its first non-RFC posting and we're rolling in a first pass at a QEMU implementation that depends on a contested kernel RFC, which depends on another stagnant kernel RFC. I'm fine with working on it in parallel, but give me some light at the end of the tunnel as a reviewer and maintainer that this code isn't going to live indefinitely on the mailing list. Do we really need those GIC patches do be able to have non-KVM accelerated VFIO platform device assignment? We certainly don't need IRQ forwarding. Thanks, Alex
Re: [Qemu-devel] [PATCH v6 00/16] KVM platform device passthrough
On Thu, Sep 11, 2014 at 04:51:14PM -0600, Alex Williamson wrote: On Thu, 2014-09-11 at 15:23 -0700, Christoffer Dall wrote: On Thu, Sep 11, 2014 at 04:14:09PM -0600, Alex Williamson wrote: On Tue, 2014-09-09 at 08:31 +0100, Eric Auger wrote: This RFC series aims at enabling KVM platform device passthrough. It implements a VFIO platform device, derived from VFIO PCI device. The VFIO platform device uses the host VFIO platform driver which must be bound to the assigned device prior to the QEMU system start. - the guest can directly access the device register space - assigned device IRQs are transparently routed to the guest by QEMU/KVM (3 methods currently are supported: user-level eventfd handling, irqfd, forwarded IRQs) - iommu is transparently programmed to prevent the device from accessing physical pages outside of the guest address space This patch series is made of the following patch files: 1-7) Modifications to PCI code to prepare for VFIO platform device 8) split of PCI specific code and generic code (move) 9-11) creation of the VFIO calxeda xgmac platform device, without irqfd support (MMIO direct access and IRQ assignment). 12) fake injection test modality (to test multiple IRQ) 13) addition of irqfd/virqfd support 14-16) forwarded IRQ Dependency List: QEMU dependencies: [1] [PATCH v2 0/9] Dynamic sysbus device allocation support, Alex Graf http://lists.gnu.org/archive/html/qemu-ppc/2014-07/msg00047.html [2] [RFC v3] machvirt dynamic sysbus device instantiation, Eric Auger [3] [PATCH v2 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html [4] [RFC] vfio: migration to trace points, Eric Auger http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00569.html Kernel Dependencies: [5] [RFC Patch v6 0/20] VFIO support for platform devices, Antonios Motakis https://www.mail-archive.com/kvm@vger.kernel.org/msg103247.html [6] [PATCH v3] ARM: KVM: add irqfd support, Eric Auger https://lkml.org/lkml/2014/9/1/141 [7] arm/arm64: KVM: Various VGIC cleanups and improvements, Christoffer Dall http://comments.gmane.org/gmane.linux.ports.arm.kernel/340430 [8] [RFC v2 0/9] KVM-VFIO IRQ forward control, Eric Auger https://lkml.org/lkml/2014/9/1/344 [9] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, Marc Zyngier http://lwn.net/Articles/603514/ kernel pieces can be found at: http://git.linaro.org/people/eric.auger/linux.git (branch 3.17rc3_irqfd_forward_integ_v2) QEMU pieces can be found at: http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v6) The patch series was tested on Calxeda Midway (ARMv7) where one xgmac is assigned to KVM host while the second one is assigned to the guest. Reworked PCI device is not tested. Wiki for Calxeda Midway setup: https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway History: v5-v6: - rebase on 2.1rc5 PCI code - forwarded IRQ first integraton Why? Are there acceleration paths that you're concerned cannot be implemented or we do not already have a proof of concept for? The base kernel patch series you depend on is 3 months old yet this series continues to grow and add new dependencies. Please let's prioritize getting something upstream instead of adding more blockers to prevent that. Thanks, I'm not exactly sure what this changelog line was referring to (depending on Marc's forwarding IRQ patches?), but just want to add that there are a number of dependencies for the GIC that need to go in as well (should happen within a few weeks), but I think it's unlikely that the IRQ forwarding stuff goes in for v3.18 at this point. It may make sense as you suggest to keep that part out of this patch set and something merged sooner as opposed to later, but I'm too jet-lagged to completely understand if that's going to be a horrible mess. The point is that we're on v6 of a patch series and its first non-RFC posting and we're rolling in a first pass at a QEMU implementation that depends on a contested kernel RFC, which depends on another stagnant kernel RFC. I'm fine with working on it in parallel, but give me some light at the end of the tunnel as a reviewer and maintainer that this code isn't going to live indefinitely on the mailing list. Do we really need those GIC patches do be able to have non-KVM accelerated VFIO platform device assignment? We certainly don't need IRQ forwarding. Thanks, You need the vgic cleanup and fixes series to do platform device assignment on ARM, yes. I would also like to see us
