Re: [Qemu-devel] [PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests
On Mon, Nov 26, 2012 at 11:46 PM, Gleb Natapov g...@redhat.com wrote: On Mon, Nov 26, 2012 at 09:46:12PM -0200, Marcelo Tosatti wrote: On Tue, Nov 20, 2012 at 02:09:46PM +, Pandarathil, Vijaymohan R wrote: -Original Message- From: Stefan Hajnoczi [mailto:stefa...@gmail.com] Sent: Tuesday, November 20, 2012 5:41 AM To: Pandarathil, Vijaymohan R Cc: kvm@vger.kernel.org; linux-...@vger.kernel.org; qemu-de...@nongnu.org; linux-ker...@vger.kernel.org Subject: Re: [PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests On Tue, Nov 20, 2012 at 06:31:48AM +, Pandarathil, Vijaymohan R wrote: Add support for error containment when a PCI pass-thru device assigned to a KVM guest encounters an error. This is for PCIe devices/drivers that support AER functionality. When the OS is notified of an error in a device either through the firmware first approach or through an interrupt handled by the AER root port driver, concerned subsystems are notified by invoking callbacks registered by these subsystems. The device is also marked as tainted till the corresponding driver recovery routines are successful. KVM module registers for a notification of such errors. In the KVM callback routine, a global counter is incremented to keep track of the error notification. Before each CPU enters guest mode to execute guest code, appropriate checks are done to see if the impacted device belongs to the guest or not. If the device belongs to the guest, qemu hypervisor for the guest is informed and the guest is immediately brought down, thus preventing or minimizing chances of any bad data being written out by the guest driver after the device has encountered an error. I'm surprised that the hypervisor would shut down the guest when PCIe AER kicks in for a pass-through device. Shouldn't we pass the AER event into the guest and deal with it there? Agreed. That would be the ideal behavior and is planned in a future patch. Lack of control over the capabilities/type of the OS/drivers running in the guest is also a concern in passing along the event to the guest. My understanding is that in the current implementation of Linux/KVM, these errors are not handled at all and can potentially cause a guest hang or crash or even data corruption depending on the implementation of the guest driver for the device. As a first step, these patches make the behavior better by doing error containment with a predictable behavior when such errors occur. For both ACPI notifications and Linux PCI AER driver there is a way for the PCI driver to receive a notification, correct? Can just have virt/kvm/assigned-dev.c code register such a notifier (as a PCI driver) and then perform appropriate action? Also the semantics of tainted driver is not entirely clear. Is there any reason for not having this feature for VFIO only, as KVM device assigment is being phased out? Exactly. We shouldn't add checks to guest entry code and introduce new userspace ABI to add minor feature to deprecated code. New userspace ABI means that QEMU changes are needed, so the feature will be fully functional only with latest QEMU which is capable of using VFIO anyway. I'm ignoring these patches for now. Please address the review comments if you think we still need to do something here. Bjorn -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests
On Tue, Nov 20, 2012 at 02:09:46PM +, Pandarathil, Vijaymohan R wrote: -Original Message- From: Stefan Hajnoczi [mailto:stefa...@gmail.com] Sent: Tuesday, November 20, 2012 5:41 AM To: Pandarathil, Vijaymohan R Cc: kvm@vger.kernel.org; linux-...@vger.kernel.org; qemu-de...@nongnu.org; linux-ker...@vger.kernel.org Subject: Re: [PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests On Tue, Nov 20, 2012 at 06:31:48AM +, Pandarathil, Vijaymohan R wrote: Add support for error containment when a PCI pass-thru device assigned to a KVM guest encounters an error. This is for PCIe devices/drivers that support AER functionality. When the OS is notified of an error in a device either through the firmware first approach or through an interrupt handled by the AER root port driver, concerned subsystems are notified by invoking callbacks registered by these subsystems. The device is also marked as tainted till the corresponding driver recovery routines are successful. KVM module registers for a notification of such errors. In the KVM callback routine, a global counter is incremented to keep track of the error notification. Before each CPU enters guest mode to execute guest code, appropriate checks are done to see if the impacted device belongs to the guest or not. If the device belongs to the guest, qemu hypervisor for the guest is informed and the guest is immediately brought down, thus preventing or minimizing chances of any bad data being written out by the guest driver after the device has encountered an error. I'm surprised that the hypervisor would shut down the guest when PCIe AER kicks in for a pass-through device. Shouldn't we pass the AER event into the guest and deal with it there? Agreed. That would be the ideal behavior and is planned in a future patch. Lack of control over the capabilities/type of the OS/drivers running in the guest is also a concern in passing along the event to the guest. My understanding is that in the current implementation of Linux/KVM, these errors are not handled at all and can potentially cause a guest hang or crash or even data corruption depending on the implementation of the guest driver for the device. As a first step, these patches make the behavior better by doing error containment with a predictable behavior when such errors occur. For both ACPI notifications and Linux PCI AER driver there is a way for the PCI driver to receive a notification, correct? Can just have virt/kvm/assigned-dev.c code register such a notifier (as a PCI driver) and then perform appropriate action? Also the semantics of tainted driver is not entirely clear. Is there any reason for not having this feature for VFIO only, as KVM device assigment is being phased out? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests
On Mon, Nov 26, 2012 at 09:46:12PM -0200, Marcelo Tosatti wrote: On Tue, Nov 20, 2012 at 02:09:46PM +, Pandarathil, Vijaymohan R wrote: -Original Message- From: Stefan Hajnoczi [mailto:stefa...@gmail.com] Sent: Tuesday, November 20, 2012 5:41 AM To: Pandarathil, Vijaymohan R Cc: kvm@vger.kernel.org; linux-...@vger.kernel.org; qemu-de...@nongnu.org; linux-ker...@vger.kernel.org Subject: Re: [PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests On Tue, Nov 20, 2012 at 06:31:48AM +, Pandarathil, Vijaymohan R wrote: Add support for error containment when a PCI pass-thru device assigned to a KVM guest encounters an error. This is for PCIe devices/drivers that support AER functionality. When the OS is notified of an error in a device either through the firmware first approach or through an interrupt handled by the AER root port driver, concerned subsystems are notified by invoking callbacks registered by these subsystems. The device is also marked as tainted till the corresponding driver recovery routines are successful. KVM module registers for a notification of such errors. In the KVM callback routine, a global counter is incremented to keep track of the error notification. Before each CPU enters guest mode to execute guest code, appropriate checks are done to see if the impacted device belongs to the guest or not. If the device belongs to the guest, qemu hypervisor for the guest is informed and the guest is immediately brought down, thus preventing or minimizing chances of any bad data being written out by the guest driver after the device has encountered an error. I'm surprised that the hypervisor would shut down the guest when PCIe AER kicks in for a pass-through device. Shouldn't we pass the AER event into the guest and deal with it there? Agreed. That would be the ideal behavior and is planned in a future patch. Lack of control over the capabilities/type of the OS/drivers running in the guest is also a concern in passing along the event to the guest. My understanding is that in the current implementation of Linux/KVM, these errors are not handled at all and can potentially cause a guest hang or crash or even data corruption depending on the implementation of the guest driver for the device. As a first step, these patches make the behavior better by doing error containment with a predictable behavior when such errors occur. For both ACPI notifications and Linux PCI AER driver there is a way for the PCI driver to receive a notification, correct? Can just have virt/kvm/assigned-dev.c code register such a notifier (as a PCI driver) and then perform appropriate action? Also the semantics of tainted driver is not entirely clear. Is there any reason for not having this feature for VFIO only, as KVM device assigment is being phased out? Exactly. We shouldn't add checks to guest entry code and introduce new userspace ABI to add minor feature to deprecated code. New userspace ABI means that QEMU changes are needed, so the feature will be fully functional only with latest QEMU which is capable of using VFIO anyway. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests
On Tue, Nov 20, 2012 at 06:31:48AM +, Pandarathil, Vijaymohan R wrote: Add support for error containment when a PCI pass-thru device assigned to a KVM guest encounters an error. This is for PCIe devices/drivers that support AER functionality. When the OS is notified of an error in a device either through the firmware first approach or through an interrupt handled by the AER root port driver, concerned subsystems are notified by invoking callbacks registered by these subsystems. The device is also marked as tainted till the corresponding driver recovery routines are successful. KVM module registers for a notification of such errors. In the KVM callback routine, a global counter is incremented to keep track of the error notification. Before each CPU enters guest mode to execute guest code, appropriate checks are done to see if the impacted device belongs to the guest or not. If the device belongs to the guest, qemu hypervisor for the guest is informed and the guest is immediately brought down, thus preventing or minimizing chances of any bad data being written out by the guest driver after the device has encountered an error. I'm surprised that the hypervisor would shut down the guest when PCIe AER kicks in for a pass-through device. Shouldn't we pass the AER event into the guest and deal with it there? The equivalent to this policy on physical hardware would be that the CPU is reset or the machine is powered down on AER. That doesn't sound right. Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests
-Original Message- From: Stefan Hajnoczi [mailto:stefa...@gmail.com] Sent: Tuesday, November 20, 2012 5:41 AM To: Pandarathil, Vijaymohan R Cc: kvm@vger.kernel.org; linux-...@vger.kernel.org; qemu-de...@nongnu.org; linux-ker...@vger.kernel.org Subject: Re: [PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests On Tue, Nov 20, 2012 at 06:31:48AM +, Pandarathil, Vijaymohan R wrote: Add support for error containment when a PCI pass-thru device assigned to a KVM guest encounters an error. This is for PCIe devices/drivers that support AER functionality. When the OS is notified of an error in a device either through the firmware first approach or through an interrupt handled by the AER root port driver, concerned subsystems are notified by invoking callbacks registered by these subsystems. The device is also marked as tainted till the corresponding driver recovery routines are successful. KVM module registers for a notification of such errors. In the KVM callback routine, a global counter is incremented to keep track of the error notification. Before each CPU enters guest mode to execute guest code, appropriate checks are done to see if the impacted device belongs to the guest or not. If the device belongs to the guest, qemu hypervisor for the guest is informed and the guest is immediately brought down, thus preventing or minimizing chances of any bad data being written out by the guest driver after the device has encountered an error. I'm surprised that the hypervisor would shut down the guest when PCIe AER kicks in for a pass-through device. Shouldn't we pass the AER event into the guest and deal with it there? Agreed. That would be the ideal behavior and is planned in a future patch. Lack of control over the capabilities/type of the OS/drivers running in the guest is also a concern in passing along the event to the guest. My understanding is that in the current implementation of Linux/KVM, these errors are not handled at all and can potentially cause a guest hang or crash or even data corruption depending on the implementation of the guest driver for the device. As a first step, these patches make the behavior better by doing error containment with a predictable behavior when such errors occur. The equivalent to this policy on physical hardware would be that the CPU is reset or the machine is powered down on AER. That doesn't sound right. Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/4] AER-KVM: Error containment of PCI pass-thru devices assigned to KVM guests
Add support for error containment when a PCI pass-thru device assigned to a KVM guest encounters an error. This is for PCIe devices/drivers that support AER functionality. When the OS is notified of an error in a device either through the firmware first approach or through an interrupt handled by the AER root port driver, concerned subsystems are notified by invoking callbacks registered by these subsystems. The device is also marked as tainted till the corresponding driver recovery routines are successful. KVM module registers for a notification of such errors. In the KVM callback routine, a global counter is incremented to keep track of the error notification. Before each CPU enters guest mode to execute guest code, appropriate checks are done to see if the impacted device belongs to the guest or not. If the device belongs to the guest, qemu hypervisor for the guest is informed and the guest is immediately brought down, thus preventing or minimizing chances of any bad data being written out by the guest driver after the device has encountered an error. Note that the changes here are specific to PCI pass-thru devices and is confined to error containment. Error recovery is not included in these set of changes. A future set of patches is planned to address SR-IOV devices and VFIO devices assigned to guests as well as recovery without bringing down the guest. --- Vijay Mohan Pandarathil(4): AER-PCI: Add infrastructure for notification of errors to other subsystems AER-GHES: Add support for error notification in firmware first approach of AER AER-KVM: Integration of KVM with AER for PCI pass-thru devices AER-QEMU: Bring down the guest when KVM detects a PCI device error arch/x86/include/asm/kvm_host.h| 1 + arch/x86/kvm/x86.c | 44 ++ drivers/acpi/apei/ghes.c | 41 +++ drivers/pci/pcie/aer/aerdrv.c | 20 + drivers/pci/pcie/aer/aerdrv_core.c | 9 +++- include/linux/aer.h| 4 include/linux/kvm_host.h | 4 include/linux/pci.h| 2 ++ include/uapi/linux/kvm.h | 1 + virt/kvm/assigned-dev.c| 34 + virt/kvm/kvm_main.c| 34 + 11 files changed, 193 insertions(+), 1 deletion(-) Qemu files changed kvm-all.c |6 ++ linux-headers/linux/kvm.h |1 + 2 files changed, 7 insertions(+) -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
