[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
This bug affects s390x only, hence I'm updating all further verification requests to done, to unblock potential ongoing processes. ** Tags removed: verification-needed-jammy-linux-mtk ** Tags added: verification-done-jammy-linux-mtk -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Released Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Hardware used: z14 or greater LPAR, PCI-attached devices (RoCE VFs, ISM devices, NVMe drive) * Setup: Both the kernel and QEMU features are needed for the feature to function (an upstream QEMU can be used to verify the kernel early), and the facility is only avaialble on z14 or newer. When any of those pieces is missing, the interpretation facility will not be used. When both the kernel and QEMU features are included in their respective packages, and running in an LPAR on a z14 or newer machine, this feature will be enabled automatically. Existing supported devices should behave as before with no changes required by an end-user (e.g. no changes to libvirt domain definitions) -- but will now make use of the interpretation facility. Additionally, ISM devices will now be eligible for vfio-pci passthrough (where before QEMU would exit on error if attempting to provide an ISM device for vfio-pci passthrough, preventing the guest from starting) * Testing will include the following scenarios, repeated each for RoCE, ISM and NVMe: 1) Testing of basic device passthrough (create a VM with a vfio-pci device as part of the libvirt domain definition, passing through a RoCE VF, an ISM device, or an NVMe drive. Verify that the device is available in the guest and functioning) 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, virsh detach-device to remove the device from the running guest, verify the device is removed from the guest, then virsh attach-device to hotplug the device to the guest again, verify the device functions in the guest) 3) Host power off testing: Power off the device from the host, verify that the device is unplugged from the guest as part of the poweroff 4) Guest power off testing: Power off the device from within the guest, verify that the device is unusuable in the guest, power the device back on within the guest and verify that the device is once again usable. 5) Guest reboot testing: (create a VM with a vfio-pci device, verify the device is in working condition, reboot the guest, verify that the device is still usable after reboot) Testing will include the following scenarios specifically for ISM devices: 1) Testing of SMC-D v1 fallback: Using 2 ISM devices on the same VCHID that share a PNETID, create 2 guests and pass one ISM device via vfio-pci device to each guest. Establish TCP connectivity between the 2 guests using the libvirt default network, and then use smc_run (https://manpages.ubuntu.com/manpages/jammy/man8/smc_run.8.html) to run an iperf workload between the 2 guests (will include both short workloads and longer-running workloads). Verify that SMC-D transfer was used between the guests instead of TCP via 'smcd stats' (https://manpages.ubuntu.com/manpages/jammy/man8/smcd.8.html) 2) Testing of SMC-D v2: Same as above, but using 2 ISM devices on the same VCHID that have no PNETID specified Testing will include the following scenarios specifically for RoCE devices: 1) Ping testing: Using 2 RoCE VFs that share a common network, create 2 guests and pass one RoCE
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
This bug is awaiting verification that the linux-mtk/5.15.0-1030.34 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-mtk' to 'verification-done-jammy- linux-mtk'. If the problem still exists, change the tag 'verification- needed-jammy-linux-mtk' to 'verification-failed-jammy-linux-mtk'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: kernel-spammed-jammy-linux-mtk-v2 verification-needed-jammy-linux-mtk -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Released Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Hardware used: z14 or greater LPAR, PCI-attached devices (RoCE VFs, ISM devices, NVMe drive) * Setup: Both the kernel and QEMU features are needed for the feature to function (an upstream QEMU can be used to verify the kernel early), and the facility is only avaialble on z14 or newer. When any of those pieces is missing, the interpretation facility will not be used. When both the kernel and QEMU features are included in their respective packages, and running in an LPAR on a z14 or newer machine, this feature will be enabled automatically. Existing supported devices should behave as before with no changes required by an end-user (e.g. no changes to libvirt domain definitions) -- but will now make use of the interpretation facility. Additionally, ISM devices will now be eligible for vfio-pci passthrough (where before QEMU would exit on error if attempting to provide an ISM device for vfio-pci passthrough, preventing the guest from starting) * Testing will include the following scenarios, repeated each for RoCE, ISM and NVMe: 1) Testing of basic device passthrough (create a VM with a vfio-pci device as part of the libvirt domain definition, passing through a RoCE VF, an ISM device, or an NVMe drive. Verify that the device is available in the guest and functioning) 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, virsh detach-device to remove the device from the running guest, verify the device is removed from the guest, then virsh attach-device to hotplug the device to the guest again, verify the device functions in the guest) 3) Host power off testing: Power off the device from the host, verify that the device is unplugged from the guest as part of the poweroff 4) Guest power off testing: Power off the device from within the guest, verify that the device is unusuable in the guest, power the device back on within the guest and verify that the device is once again usable. 5) Guest reboot testing: (create a VM with a vfio-pci device, verify the device is in working condition, reboot the guest, verify that the device is still usable after reboot) Testing will include the following scenarios specifically for ISM devices: 1) Testing of SMC-D v1 fallback: Using 2 ISM devices on the same VCHID that share a PNETID, create 2 guests and pass one ISM device via vfio-pci device to each guest. Establish TCP connectivity between the 2 guests using the libvirt default network, and then use smc_run (https://manpages.ubuntu.com/manpages/jammy/man8/smc_run.8.html) to run an iperf workload between the 2 guests (will include both
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
** Tags removed: verification-needed-focal-linux-aws-5.15 verification-needed-jammy-linux-aws verification-needed-jammy-linux-xilinx-zynqmp ** Tags added: verification-done-focal-linux-aws-5.15 verification-done-jammy-linux-aws verification-done-jammy-linux-xilinx-zynqmp -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Released Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Hardware used: z14 or greater LPAR, PCI-attached devices (RoCE VFs, ISM devices, NVMe drive) * Setup: Both the kernel and QEMU features are needed for the feature to function (an upstream QEMU can be used to verify the kernel early), and the facility is only avaialble on z14 or newer. When any of those pieces is missing, the interpretation facility will not be used. When both the kernel and QEMU features are included in their respective packages, and running in an LPAR on a z14 or newer machine, this feature will be enabled automatically. Existing supported devices should behave as before with no changes required by an end-user (e.g. no changes to libvirt domain definitions) -- but will now make use of the interpretation facility. Additionally, ISM devices will now be eligible for vfio-pci passthrough (where before QEMU would exit on error if attempting to provide an ISM device for vfio-pci passthrough, preventing the guest from starting) * Testing will include the following scenarios, repeated each for RoCE, ISM and NVMe: 1) Testing of basic device passthrough (create a VM with a vfio-pci device as part of the libvirt domain definition, passing through a RoCE VF, an ISM device, or an NVMe drive. Verify that the device is available in the guest and functioning) 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, virsh detach-device to remove the device from the running guest, verify the device is removed from the guest, then virsh attach-device to hotplug the device to the guest again, verify the device functions in the guest) 3) Host power off testing: Power off the device from the host, verify that the device is unplugged from the guest as part of the poweroff 4) Guest power off testing: Power off the device from within the guest, verify that the device is unusuable in the guest, power the device back on within the guest and verify that the device is once again usable. 5) Guest reboot testing: (create a VM with a vfio-pci device, verify the device is in working condition, reboot the guest, verify that the device is still usable after reboot) Testing will include the following scenarios specifically for ISM devices: 1) Testing of SMC-D v1 fallback: Using 2 ISM devices on the same VCHID that share a PNETID, create 2 guests and pass one ISM device via vfio-pci device to each guest. Establish TCP connectivity between the 2 guests using the libvirt default network, and then use smc_run (https://manpages.ubuntu.com/manpages/jammy/man8/smc_run.8.html) to run an iperf workload between the 2 guests (will include both short workloads and longer-running workloads). Verify that SMC-D transfer was used between the guests instead of TCP via 'smcd stats' (https://manpages.ubuntu.com/manpages/jammy/man8/smcd.8.html) 2) Testing of SMC-D v2: Same as above, but using 2 ISM devices on the same VCHID that have no PNETID specified Testing will include the following scenarios specifically for RoCE devices: 1) Ping testing: Using 2 RoCE VFs that share a common
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
This bug is awaiting verification that the linux-xilinx- zynqmp/5.15.0-1024.28 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-xilinx-zynqmp' to 'verification-done-jammy-linux-xilinx-zynqmp'. If the problem still exists, change the tag 'verification-needed-jammy-linux-xilinx-zynqmp' to 'verification-failed-jammy-linux-xilinx-zynqmp'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: kernel-spammed-jammy-linux-xilinx-zynqmp-v2 verification-needed-jammy-linux-xilinx-zynqmp -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Released Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Hardware used: z14 or greater LPAR, PCI-attached devices (RoCE VFs, ISM devices, NVMe drive) * Setup: Both the kernel and QEMU features are needed for the feature to function (an upstream QEMU can be used to verify the kernel early), and the facility is only avaialble on z14 or newer. When any of those pieces is missing, the interpretation facility will not be used. When both the kernel and QEMU features are included in their respective packages, and running in an LPAR on a z14 or newer machine, this feature will be enabled automatically. Existing supported devices should behave as before with no changes required by an end-user (e.g. no changes to libvirt domain definitions) -- but will now make use of the interpretation facility. Additionally, ISM devices will now be eligible for vfio-pci passthrough (where before QEMU would exit on error if attempting to provide an ISM device for vfio-pci passthrough, preventing the guest from starting) * Testing will include the following scenarios, repeated each for RoCE, ISM and NVMe: 1) Testing of basic device passthrough (create a VM with a vfio-pci device as part of the libvirt domain definition, passing through a RoCE VF, an ISM device, or an NVMe drive. Verify that the device is available in the guest and functioning) 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, virsh detach-device to remove the device from the running guest, verify the device is removed from the guest, then virsh attach-device to hotplug the device to the guest again, verify the device functions in the guest) 3) Host power off testing: Power off the device from the host, verify that the device is unplugged from the guest as part of the poweroff 4) Guest power off testing: Power off the device from within the guest, verify that the device is unusuable in the guest, power the device back on within the guest and verify that the device is once again usable. 5) Guest reboot testing: (create a VM with a vfio-pci device, verify the device is in working condition, reboot the guest, verify that the device is still usable after reboot) Testing will include the following scenarios specifically for ISM devices: 1) Testing of SMC-D v1 fallback: Using 2 ISM devices on the same VCHID that share a PNETID, create 2 guests and pass one ISM device via vfio-pci device to each guest. Establish TCP connectivity between the 2 guests using the libvirt default network, and then use smc_run (https://manpages.ubuntu.com/manpages/jammy/man8/smc_run.8.html)
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
This bug is awaiting verification that the linux- aws-5.15/5.15.0-1046.51~20.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal-linux- aws-5.15' to 'verification-done-focal-linux-aws-5.15'. If the problem still exists, change the tag 'verification-needed-focal-linux-aws-5.15' to 'verification-failed-focal-linux-aws-5.15'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: kernel-spammed-focal-linux-aws-5.15-v2 verification-needed-focal-linux-aws-5.15 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Released Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Hardware used: z14 or greater LPAR, PCI-attached devices (RoCE VFs, ISM devices, NVMe drive) * Setup: Both the kernel and QEMU features are needed for the feature to function (an upstream QEMU can be used to verify the kernel early), and the facility is only avaialble on z14 or newer. When any of those pieces is missing, the interpretation facility will not be used. When both the kernel and QEMU features are included in their respective packages, and running in an LPAR on a z14 or newer machine, this feature will be enabled automatically. Existing supported devices should behave as before with no changes required by an end-user (e.g. no changes to libvirt domain definitions) -- but will now make use of the interpretation facility. Additionally, ISM devices will now be eligible for vfio-pci passthrough (where before QEMU would exit on error if attempting to provide an ISM device for vfio-pci passthrough, preventing the guest from starting) * Testing will include the following scenarios, repeated each for RoCE, ISM and NVMe: 1) Testing of basic device passthrough (create a VM with a vfio-pci device as part of the libvirt domain definition, passing through a RoCE VF, an ISM device, or an NVMe drive. Verify that the device is available in the guest and functioning) 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, virsh detach-device to remove the device from the running guest, verify the device is removed from the guest, then virsh attach-device to hotplug the device to the guest again, verify the device functions in the guest) 3) Host power off testing: Power off the device from the host, verify that the device is unplugged from the guest as part of the poweroff 4) Guest power off testing: Power off the device from within the guest, verify that the device is unusuable in the guest, power the device back on within the guest and verify that the device is once again usable. 5) Guest reboot testing: (create a VM with a vfio-pci device, verify the device is in working condition, reboot the guest, verify that the device is still usable after reboot) Testing will include the following scenarios specifically for ISM devices: 1) Testing of SMC-D v1 fallback: Using 2 ISM devices on the same VCHID that share a PNETID, create 2 guests and pass one ISM device via vfio-pci device to each guest. Establish TCP connectivity between the 2 guests using the libvirt default network, and then use smc_run (https://manpages.ubuntu.com/manpages/jammy/man8/smc_run.8.html) to run an iperf workload
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
--- Comment From boris.m...@de.ibm.com 2023-09-01 10:31 EDT--- As this has been released to jammy -updates, we can close the bug. Thanks everyone for all your work. ==> Changing the status to "CLOSED" ** Tags removed: verification-needed-jammy-linux-azure -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Released Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Hardware used: z14 or greater LPAR, PCI-attached devices (RoCE VFs, ISM devices, NVMe drive) * Setup: Both the kernel and QEMU features are needed for the feature to function (an upstream QEMU can be used to verify the kernel early), and the facility is only avaialble on z14 or newer. When any of those pieces is missing, the interpretation facility will not be used. When both the kernel and QEMU features are included in their respective packages, and running in an LPAR on a z14 or newer machine, this feature will be enabled automatically. Existing supported devices should behave as before with no changes required by an end-user (e.g. no changes to libvirt domain definitions) -- but will now make use of the interpretation facility. Additionally, ISM devices will now be eligible for vfio-pci passthrough (where before QEMU would exit on error if attempting to provide an ISM device for vfio-pci passthrough, preventing the guest from starting) * Testing will include the following scenarios, repeated each for RoCE, ISM and NVMe: 1) Testing of basic device passthrough (create a VM with a vfio-pci device as part of the libvirt domain definition, passing through a RoCE VF, an ISM device, or an NVMe drive. Verify that the device is available in the guest and functioning) 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, virsh detach-device to remove the device from the running guest, verify the device is removed from the guest, then virsh attach-device to hotplug the device to the guest again, verify the device functions in the guest) 3) Host power off testing: Power off the device from the host, verify that the device is unplugged from the guest as part of the poweroff 4) Guest power off testing: Power off the device from within the guest, verify that the device is unusuable in the guest, power the device back on within the guest and verify that the device is once again usable. 5) Guest reboot testing: (create a VM with a vfio-pci device, verify the device is in working condition, reboot the guest, verify that the device is still usable after reboot) Testing will include the following scenarios specifically for ISM devices: 1) Testing of SMC-D v1 fallback: Using 2 ISM devices on the same VCHID that share a PNETID, create 2 guests and pass one ISM device via vfio-pci device to each guest. Establish TCP connectivity between the 2 guests using the libvirt default network, and then use smc_run (https://manpages.ubuntu.com/manpages/jammy/man8/smc_run.8.html) to run an iperf workload between the 2 guests (will include both short workloads and longer-running workloads). Verify that SMC-D transfer was used between the guests instead of TCP via 'smcd stats' (https://manpages.ubuntu.com/manpages/jammy/man8/smcd.8.html) 2) Testing of SMC-D v2: Same as above, but using 2 ISM devices on the same VCHID that have no PNETID specified Testing will include the following scenarios specifically for RoCE devices: 1) Ping testing: Using 2 RoCE VFs that share a common network,
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
@mjrosato Many thx for this successful kernel verification! (I'll cross-reference this as fyi to the qemu bug, too.) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Committed Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Hardware used: z14 or greater LPAR, PCI-attached devices (RoCE VFs, ISM devices, NVMe drive) * Setup: Both the kernel and QEMU features are needed for the feature to function (an upstream QEMU can be used to verify the kernel early), and the facility is only avaialble on z14 or newer. When any of those pieces is missing, the interpretation facility will not be used. When both the kernel and QEMU features are included in their respective packages, and running in an LPAR on a z14 or newer machine, this feature will be enabled automatically. Existing supported devices should behave as before with no changes required by an end-user (e.g. no changes to libvirt domain definitions) -- but will now make use of the interpretation facility. Additionally, ISM devices will now be eligible for vfio-pci passthrough (where before QEMU would exit on error if attempting to provide an ISM device for vfio-pci passthrough, preventing the guest from starting) * Testing will include the following scenarios, repeated each for RoCE, ISM and NVMe: 1) Testing of basic device passthrough (create a VM with a vfio-pci device as part of the libvirt domain definition, passing through a RoCE VF, an ISM device, or an NVMe drive. Verify that the device is available in the guest and functioning) 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, virsh detach-device to remove the device from the running guest, verify the device is removed from the guest, then virsh attach-device to hotplug the device to the guest again, verify the device functions in the guest) 3) Host power off testing: Power off the device from the host, verify that the device is unplugged from the guest as part of the poweroff 4) Guest power off testing: Power off the device from within the guest, verify that the device is unusuable in the guest, power the device back on within the guest and verify that the device is once again usable. 5) Guest reboot testing: (create a VM with a vfio-pci device, verify the device is in working condition, reboot the guest, verify that the device is still usable after reboot) Testing will include the following scenarios specifically for ISM devices: 1) Testing of SMC-D v1 fallback: Using 2 ISM devices on the same VCHID that share a PNETID, create 2 guests and pass one ISM device via vfio-pci device to each guest. Establish TCP connectivity between the 2 guests using the libvirt default network, and then use smc_run (https://manpages.ubuntu.com/manpages/jammy/man8/smc_run.8.html) to run an iperf workload between the 2 guests (will include both short workloads and longer-running workloads). Verify that SMC-D transfer was used between the guests instead of TCP via 'smcd stats' (https://manpages.ubuntu.com/manpages/jammy/man8/smcd.8.html) 2) Testing of SMC-D v2: Same as above, but using 2 ISM devices on the same VCHID that have no PNETID specified Testing will include the following scenarios specifically for RoCE devices: 1) Ping testing: Using 2 RoCE VFs that share a common network, create 2 guests and pass one RoCE device to each guest. Assign IP addresses within each guest to the associated TCP interface, perform a
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
--- Comment From mjros...@us.ibm.com 2023-07-20 13:50 EDT--- Verification of this kernel feature is complete. All tests were run using the kernel in jammy-proposed (5.15.0-79-generic #86-Ubuntu) on the host as well as in the KVM guest (except where otherwise noted, see below). Because the necessary qemu changes are not yet in jammy-proposed, I used a custom qemu build based on the launchpad qemu repository + applied the QEMU feature backport. I will do a second verification of the QEMU feature once that package is available. As for the testing itself: NVMe and RoCE testing went just as expected. During ISM testing, I did run into one issue that is not caused by this feature (therefore I am marking verification complete) but it is something that will need to be fixed separately once an upstream patch is available. Specifically, when using the jammy-proposed kernel in the KVM guest I noted occasional UBSAN array-index-out-of-bounds warnings from net/smc/af_smc.c -- This is not directly due to the code added by this feature or even due to being run in a KVM guest, but rather a bug in net/smc code (specifically the SMC_STAT_PAYLOAD_SUB macro) that is much more obvious when ubsan is enabled. I've already identified the root cause, verified that it also exists upstream, and reported the issue to the SMC maintainers along with a suggested fix - without UBSAN enabled no warning is issued but an incorrect performance status counter is quietly incremented. I had panic_on_warn enabled in the KVM guest, therefore the warning caused the guest to crash; otherwise I likely would not have noticed this issue at all. In order to complete all verification tests related to ISM devices, I proceeded with 2 different KVM guest kernel configurations and ran all ISM tests using both: 1) a custom built (upstream 6.4.0) guest kernel that did not include UBSAN 2) the jammy-proposed kernel that includes UBSAN but with panic_on_warn=0 All test scenarios passed with these guest kernels. ** Tags removed: verification-needed-jammy ** Tags added: verification-done-jammy -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Committed Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Hardware used: z14 or greater LPAR, PCI-attached devices (RoCE VFs, ISM devices, NVMe drive) * Setup: Both the kernel and QEMU features are needed for the feature to function (an upstream QEMU can be used to verify the kernel early), and the facility is only avaialble on z14 or newer. When any of those pieces is missing, the interpretation facility will not be used. When both the kernel and QEMU features are included in their respective packages, and running in an LPAR on a z14 or newer machine, this feature will be enabled automatically. Existing supported devices should behave as before with no changes required by an end-user (e.g. no changes to libvirt domain definitions) -- but will now make use of the interpretation facility. Additionally, ISM devices will now be eligible for vfio-pci passthrough (where before QEMU would exit on error if attempting to provide an ISM device for vfio-pci passthrough, preventing the guest from starting) * Testing will include the following scenarios, repeated each for RoCE, ISM and NVMe: 1) Testing of basic device passthrough (create a VM with a vfio-pci device as part of the libvirt domain definition, passing through a RoCE VF, an ISM device, or an NVMe drive. Verify that the device is available in the guest and functioning) 2) Testing of device hotplug/unplug (create a VM with a vfio-pci
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
** Description changed: [ Impact ] - * Currently the PCI passthrough implementation for s390x is based on -intercepting PCI I/O instructions, which leads to a reduced I/O performance -compared to the execution of PCI instructions directly in LPAR. + * Currently the PCI passthrough implementation for s390x is based on + intercepting PCI I/O instructions, which leads to a reduced I/O performance + compared to the execution of PCI instructions directly in LPAR. - * Hence users may face I/O bottlenecks when using PCI devices in passthrough -mode based on the current implementation. + * Hence users may face I/O bottlenecks when using PCI devices in passthrough + mode based on the current implementation. - * For avoiding this and to improve performance, the interpretive execution -of the PCI store and PCI load instructions get enabled. + * For avoiding this and to improve performance, the interpretive execution + of the PCI store and PCI load instructions get enabled. - * A further improvement is achieved by enabling the Adapter-Event-Notification -Interpretation (AENI). + * A further improvement is achieved by enabling the Adapter-Event-Notification + Interpretation (AENI). - * Since LTS releases are the main focus for stable and long running KVM -workloads, it is highly desired to get this backported to the jammy kernel -(and because the next LTS is still some time away). + * Since LTS releases are the main focus for stable and long running KVM + workloads, it is highly desired to get this backported to the jammy kernel + (and because the next LTS is still some time away). [ Test Plan ] - * Have an Ubuntu Server 22.04 installation on LPAR, -that is able to access (ideally multiple) PCI devices, -like RoCE Express (network) or NVMe disks. + * Hardware used: z14 or greater LPAR, PCI-attached devices + (RoCE VFs, ISM devices, NVMe drive) - * Setup KVM and pass through (ideally multiple) of these PCI devices -(that are otherwise unused on the KVM host). + * Setup: Both the kernel and QEMU features are needed for the feature + to function (an upstream QEMU can be used to verify the kernel early), + and the facility is only avaialble on z14 or newer. + When any of those pieces is missing, + the interpretation facility will not be used. + When both the kernel and QEMU features are included in their respective + packages, and running in an LPAR on a z14 or newer machine, + this feature will be enabled automatically. + Existing supported devices should behave as before with no changes + required by an end-user (e.g. no changes to libvirt domain definitions) + -- but will now make use of the interpretation facility. + Additionally, ISM devices will now be eligible for vfio-pci passthrough + (where before QEMU would exit on error if attempting to provide an ISM + device for vfio-pci passthrough, preventing the guest from starting) - * Generate IO load on these passed through PCI devices, -for example with stress-ng, using class network and/or device -and/or io stressors. + * Testing will include the following scenarios, repeated each for RoCE, + ISM and NVMe: - * This PR also introduces a new kernel config option 'VFIO_PCI_ZDEV_KVM' -that allows to enable support for the s390x-specific extensions and -enhancements to KVM passthrough, such as interpretive execution of zPCI -instructions and is with this PR and got enabled. + 1) Testing of basic device passthrough (create a VM with a vfio-pci + device as part of the libvirt domain definition, passing through + a RoCE VF, an ISM device, or an NVMe drive. Verify that the device + is available in the guest and functioning) + 2) Testing of device hotplug/unplug (create a VM with a vfio-pci device, + virsh detach-device to remove the device from the running guest, + verify the device is removed from the guest, then virsh attach-device + to hotplug the device to the guest again, verify the device functions + in the guest) + 3) Host power off testing: Power off the device from the host, verify + that the device is unplugged from the guest as part of the poweroff + 4) Guest power off testing: Power off the device from within the guest, + verify that the device is unusuable in the guest, + power the device back on within the guest and verify that the device + is once again usable. + 5) Guest reboot testing: (create a VM with a vfio-pci device, + verify the device is in working condition, reboot the guest, + verify that the device is still usable after reboot) - * The qemu autopkgtest (also needed due to LP#1853307) will be a got fit -to identify any potential regressions, also in the kvm kernel area. + Testing will include the following scenarios specifically for ISM + devices: - * zPCI passthrough related test will be done by IBM. + 1)
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
@mjrosato - ok, please take a few more days. But what's needed is a detailed test plan about how this (L#1853306), in combination with the qemu part (LP#1853307 - see commet #23), that shows how this be verified. It should be detailed, step-by-step and e2e - it's needed for this (actually every) SRU and requested by the SRU team, which is responsible for the final acceptance. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Committed Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Have an Ubuntu Server 22.04 installation on LPAR, that is able to access (ideally multiple) PCI devices, like RoCE Express (network) or NVMe disks. * Setup KVM and pass through (ideally multiple) of these PCI devices (that are otherwise unused on the KVM host). * Generate IO load on these passed through PCI devices, for example with stress-ng, using class network and/or device and/or io stressors. * This PR also introduces a new kernel config option 'VFIO_PCI_ZDEV_KVM' that allows to enable support for the s390x-specific extensions and enhancements to KVM passthrough, such as interpretive execution of zPCI instructions and is with this PR and got enabled. * The qemu autopkgtest (also needed due to LP#1853307) will be a got fit to identify any potential regressions, also in the kvm kernel area. * zPCI passthrough related test will be done by IBM. [ Where problems could occur ] * The modifications do not change the way users or APIs have to make use of PCI passthrough, only the internal implementation got modified. * The vast majority of the code changes/or additional code is s390x-specific, under arch/s390 and drivers/s390. * However there is also common code touched: * 'kvm: use kvfree() in kvm_arch_free_vm()' touches arch/arm64/include/asm/kvm_host.h, arch/arm64/kvm/arm.c, arch/x86/include/asm/kvm_host.h, arch/x86/kvm/x86.c, include/linux/kvm_host.h switches in kvm_arch_free_vm() from kfree() to kvfree() allowing to use the common variant, which is upstream since v5.16 and with that well established. * And 'vfio-pci/zdev: add open/close device hooks' touches drivers/vfio/pci/vfio_pci_core.c and drivers/vfio/pci/vfio_pci_zdev.c include/linux/vfio_pci_core.h add now code to introduce device hooks. It's upstream since kernel 6.0. * 'KVM: s390: pci: provide routines for en-/disabling interrupt forwarding' expands a single #if statement in include/linux/sched/user.h. * 'KVM: s390: add KVM_S390_ZPCI_OP to manage guest zPCI devices' adds s390x specific KVM_S390_ZPCI_OP and it's definition to include/uapi/linux/kvm.h. * And 'vfio-pci/zdev: different maxstbl for interpreted devices' and 'vfio-pci/zdev: add function handle to clp base capability' expand s390x-specific (aka z-specific aka zdev) device structs in include/uapi/linux/vfio_zdev.h. * This shows that the vast majority of modifications are s390x specific, even in most of the common code files. * The remaining modifications in the (generally) common code files are related to the newly introduced kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM' and documentation. * The s390x changes are more significant, and could not only harm passthrough itself for zPCI devices, but also KVM virtualization in general. * In addition to these kernel changes, qemu modifications are needed as well (that are addressed at LP#1853307), this modified kernel must be tested in combination with the updated qemu package. - The qemu autopkgtest will be a
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
This bug is awaiting verification that the linux/5.15.0-79.86 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: kernel-spammed-jammy-linux verification-needed-jammy -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Committed Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Have an Ubuntu Server 22.04 installation on LPAR, that is able to access (ideally multiple) PCI devices, like RoCE Express (network) or NVMe disks. * Setup KVM and pass through (ideally multiple) of these PCI devices (that are otherwise unused on the KVM host). * Generate IO load on these passed through PCI devices, for example with stress-ng, using class network and/or device and/or io stressors. * This PR also introduces a new kernel config option 'VFIO_PCI_ZDEV_KVM' that allows to enable support for the s390x-specific extensions and enhancements to KVM passthrough, such as interpretive execution of zPCI instructions and is with this PR and got enabled. * The qemu autopkgtest (also needed due to LP#1853307) will be a got fit to identify any potential regressions, also in the kvm kernel area. * zPCI passthrough related test will be done by IBM. [ Where problems could occur ] * The modifications do not change the way users or APIs have to make use of PCI passthrough, only the internal implementation got modified. * The vast majority of the code changes/or additional code is s390x-specific, under arch/s390 and drivers/s390. * However there is also common code touched: * 'kvm: use kvfree() in kvm_arch_free_vm()' touches arch/arm64/include/asm/kvm_host.h, arch/arm64/kvm/arm.c, arch/x86/include/asm/kvm_host.h, arch/x86/kvm/x86.c, include/linux/kvm_host.h switches in kvm_arch_free_vm() from kfree() to kvfree() allowing to use the common variant, which is upstream since v5.16 and with that well established. * And 'vfio-pci/zdev: add open/close device hooks' touches drivers/vfio/pci/vfio_pci_core.c and drivers/vfio/pci/vfio_pci_zdev.c include/linux/vfio_pci_core.h add now code to introduce device hooks. It's upstream since kernel 6.0. * 'KVM: s390: pci: provide routines for en-/disabling interrupt forwarding' expands a single #if statement in include/linux/sched/user.h. * 'KVM: s390: add KVM_S390_ZPCI_OP to manage guest zPCI devices' adds s390x specific KVM_S390_ZPCI_OP and it's definition to include/uapi/linux/kvm.h. * And 'vfio-pci/zdev: different maxstbl for interpreted devices' and 'vfio-pci/zdev: add function handle to clp base capability' expand s390x-specific (aka z-specific aka zdev) device structs in include/uapi/linux/vfio_zdev.h. * This shows that the vast majority of modifications are s390x specific, even in most of the common code files. * The remaining modifications in the (generally) common code files are related to the newly introduced kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM' and documentation. * The s390x changes are more significant, and could not only harm passthrough itself for zPCI
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
** Changed in: linux (Ubuntu Jammy) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: Fix Committed Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Have an Ubuntu Server 22.04 installation on LPAR, that is able to access (ideally multiple) PCI devices, like RoCE Express (network) or NVMe disks. * Setup KVM and pass through (ideally multiple) of these PCI devices (that are otherwise unused on the KVM host). * Generate IO load on these passed through PCI devices, for example with stress-ng, using class network and/or device and/or io stressors. * This PR also introduces a new kernel config option 'VFIO_PCI_ZDEV_KVM' that allows to enable support for the s390x-specific extensions and enhancements to KVM passthrough, such as interpretive execution of zPCI instructions and is with this PR and got enabled. * The qemu autopkgtest (also needed due to LP#1853307) will be a got fit to identify any potential regressions, also in the kvm kernel area. * zPCI passthrough related test will be done by IBM. [ Where problems could occur ] * The modifications do not change the way users or APIs have to make use of PCI passthrough, only the internal implementation got modified. * The vast majority of the code changes/or additional code is s390x-specific, under arch/s390 and drivers/s390. * However there is also common code touched: * 'kvm: use kvfree() in kvm_arch_free_vm()' touches arch/arm64/include/asm/kvm_host.h, arch/arm64/kvm/arm.c, arch/x86/include/asm/kvm_host.h, arch/x86/kvm/x86.c, include/linux/kvm_host.h switches in kvm_arch_free_vm() from kfree() to kvfree() allowing to use the common variant, which is upstream since v5.16 and with that well established. * And 'vfio-pci/zdev: add open/close device hooks' touches drivers/vfio/pci/vfio_pci_core.c and drivers/vfio/pci/vfio_pci_zdev.c include/linux/vfio_pci_core.h add now code to introduce device hooks. It's upstream since kernel 6.0. * 'KVM: s390: pci: provide routines for en-/disabling interrupt forwarding' expands a single #if statement in include/linux/sched/user.h. * 'KVM: s390: add KVM_S390_ZPCI_OP to manage guest zPCI devices' adds s390x specific KVM_S390_ZPCI_OP and it's definition to include/uapi/linux/kvm.h. * And 'vfio-pci/zdev: different maxstbl for interpreted devices' and 'vfio-pci/zdev: add function handle to clp base capability' expand s390x-specific (aka z-specific aka zdev) device structs in include/uapi/linux/vfio_zdev.h. * This shows that the vast majority of modifications are s390x specific, even in most of the common code files. * The remaining modifications in the (generally) common code files are related to the newly introduced kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM' and documentation. * The s390x changes are more significant, and could not only harm passthrough itself for zPCI devices, but also KVM virtualization in general. * In addition to these kernel changes, qemu modifications are needed as well (that are addressed at LP#1853307), this modified kernel must be tested in combination with the updated qemu package. - The qemu autopkgtest will be a got fit to identify any regressions, also in the kernel. - In addition some passthrough related test will be done by IBM __ The PCI Passthrough implementation is based on intercepting PCI I/O instructions which leads to a reduced I/O performance compared to execution of PCI
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
Pull request submitted to kernel team's mailing list: https://lists.ubuntu.com/archives/kernel-team/2023-June/thread.html#140405 changing status to 'In Progress'. A test kernel was build in PPA and is available here: https://launchpad.net/~fheimes/+archive/ubuntu/lp1853306 ** Changed in: linux (Ubuntu Jammy) Assignee: (unassigned) => Canonical Kernel Team (canonical-kernel-team) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: In Progress Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: [ Impact ] * Currently the PCI passthrough implementation for s390x is based on intercepting PCI I/O instructions, which leads to a reduced I/O performance compared to the execution of PCI instructions directly in LPAR. * Hence users may face I/O bottlenecks when using PCI devices in passthrough mode based on the current implementation. * For avoiding this and to improve performance, the interpretive execution of the PCI store and PCI load instructions get enabled. * A further improvement is achieved by enabling the Adapter-Event-Notification Interpretation (AENI). * Since LTS releases are the main focus for stable and long running KVM workloads, it is highly desired to get this backported to the jammy kernel (and because the next LTS is still some time away). [ Test Plan ] * Have an Ubuntu Server 22.04 installation on LPAR, that is able to access (ideally multiple) PCI devices, like RoCE Express (network) or NVMe disks. * Setup KVM and pass through (ideally multiple) of these PCI devices (that are otherwise unused on the KVM host). * Generate IO load on these passed through PCI devices, for example with stress-ng, using class network and/or device and/or io stressors. * This PR also introduces a new kernel config option 'VFIO_PCI_ZDEV_KVM' that allows to enable support for the s390x-specific extensions and enhancements to KVM passthrough, such as interpretive execution of zPCI instructions and is with this PR and got enabled. * The qemu autopkgtest (also needed due to LP#1853307) will be a got fit to identify any potential regressions, also in the kvm kernel area. * zPCI passthrough related test will be done by IBM. [ Where problems could occur ] * The modifications do not change the way users or APIs have to make use of PCI passthrough, only the internal implementation got modified. * The vast majority of the code changes/or additional code is s390x-specific, under arch/s390 and drivers/s390. * However there is also common code touched: * 'kvm: use kvfree() in kvm_arch_free_vm()' touches arch/arm64/include/asm/kvm_host.h, arch/arm64/kvm/arm.c, arch/x86/include/asm/kvm_host.h, arch/x86/kvm/x86.c, include/linux/kvm_host.h switches in kvm_arch_free_vm() from kfree() to kvfree() allowing to use the common variant, which is upstream since v5.16 and with that well established. * And 'vfio-pci/zdev: add open/close device hooks' touches drivers/vfio/pci/vfio_pci_core.c and drivers/vfio/pci/vfio_pci_zdev.c include/linux/vfio_pci_core.h add now code to introduce device hooks. It's upstream since kernel 6.0. * 'KVM: s390: pci: provide routines for en-/disabling interrupt forwarding' expands a single #if statement in include/linux/sched/user.h. * 'KVM: s390: add KVM_S390_ZPCI_OP to manage guest zPCI devices' adds s390x specific KVM_S390_ZPCI_OP and it's definition to include/uapi/linux/kvm.h. * And 'vfio-pci/zdev: different maxstbl for interpreted devices' and 'vfio-pci/zdev: add function handle to clp base capability' expand s390x-specific (aka z-specific aka zdev) device structs in include/uapi/linux/vfio_zdev.h. * This shows that the vast majority of modifications are s390x specific, even in most of the common code files. * The remaining modifications in the (generally) common code files are related to the newly introduced kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM' and documentation. * The s390x changes are more significant, and could not only harm passthrough itself for zPCI devices, but also KVM virtualization in general. * In addition to these kernel changes, qemu modifications are needed as well (that are addressed at LP#1853307), this modified kernel must be tested in combination with the updated qemu package. - The qemu autopkgtest will be a
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
** Description changed: - The PCI Passthrough implementation is based on intercepting PCI I/O instructions which leads to a reduced I/O performance compared to execution of PCI instructions in LPAR. - For improved performance the interpretive execution of the PCI store and PCI load instructions get enabled. + [ Impact ] + + * Currently the PCI passthrough implementation for s390x is based on +intercepting PCI I/O instructions, which leads to a reduced I/O performance +compared to the execution of PCI instructions directly in LPAR. + + * Hence users may face I/O bottlenecks when using PCI devices in passthrough +mode based on the current implementation. + + * For avoiding this and to improve performance, the interpretive execution +of the PCI store and PCI load instructions get enabled. + + * A further improvement is achieved by enabling the Adapter-Event-Notification +Interpretation (AENI). + + * Since LTS releases are the main focus for stable and long running KVM +workloads, it is highly desired to get this backported to the jammy kernel +(and because the next LTS is still some time away). + + [ Test Plan ] + + * Have an Ubuntu Server 22.04 installation on LPAR, +that is able to access (ideally multiple) PCI devices, +like RoCE Express (network) or NVMe disks. + + * Setup KVM and pass through (ideally multiple) of these PCI devices +(that are otherwise unused on the KVM host). + + * Generate IO load on these passed through PCI devices, +for example with stress-ng, using class network and/or device +and/or io stressors. + + * This PR also introduces a new kernel config option 'VFIO_PCI_ZDEV_KVM' +that allows to enable support for the s390x-specific extensions and +enhancements to KVM passthrough, such as interpretive execution of zPCI +instructions and is with this PR and got enabled. + + * The qemu autopkgtest (also needed due to LP#1853307) will be a got fit +to identify any potential regressions, also in the kvm kernel area. + + * zPCI passthrough related test will be done by IBM. + + + [ Where problems could occur ] + + * The modifications do not change the way users or APIs have to make +use of PCI passthrough, only the internal implementation got modified. + + * The vast majority of the code changes/or additional code is s390x-specific, +under arch/s390 and drivers/s390. + + * However there is also common code touched: + + * 'kvm: use kvfree() in kvm_arch_free_vm()' touches +arch/arm64/include/asm/kvm_host.h, arch/arm64/kvm/arm.c, +arch/x86/include/asm/kvm_host.h, arch/x86/kvm/x86.c, +include/linux/kvm_host.h switches in kvm_arch_free_vm() from kfree() to +kvfree() allowing to use the common variant, which is upstream since v5.16 +and with that well established. + + * And 'vfio-pci/zdev: add open/close device hooks' touches +drivers/vfio/pci/vfio_pci_core.c and drivers/vfio/pci/vfio_pci_zdev.c +include/linux/vfio_pci_core.h add now code to introduce device hooks. +It's upstream since kernel 6.0. + + * 'KVM: s390: pci: provide routines for en-/disabling interrupt forwarding' +expands a single #if statement in include/linux/sched/user.h. + + * 'KVM: s390: add KVM_S390_ZPCI_OP to manage guest zPCI devices' +adds s390x specific KVM_S390_ZPCI_OP and it's definition to +include/uapi/linux/kvm.h. + + * And 'vfio-pci/zdev: different maxstbl for interpreted devices' and +'vfio-pci/zdev: add function handle to clp base capability' expand +s390x-specific (aka z-specific aka zdev) device structs in +include/uapi/linux/vfio_zdev.h. + + * This shows that the vast majority of modifications are s390x specific, +even in most of the common code files. + + * The remaining modifications in the (generally) common code files are +related to the newly introduced kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM' +and documentation. + + * The s390x changes are more significant, and could not only harm +passthrough itself for zPCI devices, but also KVM virtualization in general. + + * In addition to these kernel changes, qemu modifications are needed +as well (that are addressed at LP#1853307), this modified kernel +must be tested in combination with the updated qemu package. +- The qemu autopkgtest will be a got fit to identify any regressions, +also in the kernel. +- In addition some passthrough related test will be done by IBM + + __ + + The PCI Passthrough implementation is based on intercepting PCI I/O instructions which leads to a reduced I/O performance compared to execution of PCI instructions in LPAR. + For improved performance the interpretive execution of the PCI store and PCI load instructions get enabled. Further improvement is achieved by enabling the Adapter-Event-Notification -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
PR for the above 30 commits requested for jammy, plus setting a default for a new kernel option ** Patch added: "-pull.patch" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1853306/+attachment/5679544/+files/-pull.patch -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: In Progress Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: The PCI Passthrough implementation is based on intercepting PCI I/O instructions which leads to a reduced I/O performance compared to execution of PCI instructions in LPAR. For improved performance the interpretive execution of the PCI store and PCI load instructions get enabled. Further improvement is achieved by enabling the Adapter-Event-Notification To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1853306/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
Test kernels (for all major archs, for regression testing) for jammy/22.04 (with VFIO zPCI pass-through for s390x enabled, so CONFIG_VFIO_PCI_ZDEV_KVM=y, means build in, not as module) are currently building here: https://launchpad.net/~fheimes/+archive/ubuntu/lp1853306 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: In Progress Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: The PCI Passthrough implementation is based on intercepting PCI I/O instructions which leads to a reduced I/O performance compared to execution of PCI instructions in LPAR. For improved performance the interpretive execution of the PCI store and PCI load instructions get enabled. Further improvement is achieved by enabling the Adapter-Event-Notification To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1853306/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
** Also affects: linux (Ubuntu Kinetic) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Mantic) Importance: Medium Assignee: Canonical Kernel Team (canonical-kernel-team) Status: Fix Released ** Also affects: linux (Ubuntu Lunar) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Jammy) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Lunar) Status: New => Fix Released ** Changed in: linux (Ubuntu Kinetic) Status: New => Won't Fix ** Changed in: linux (Ubuntu Jammy) Status: New => In Progress -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Jammy: In Progress Status in linux source package in Kinetic: Won't Fix Status in linux source package in Lunar: Fix Released Status in linux source package in Mantic: Fix Released Bug description: The PCI Passthrough implementation is based on intercepting PCI I/O instructions which leads to a reduced I/O performance compared to execution of PCI instructions in LPAR. For improved performance the interpretive execution of the PCI store and PCI load instructions get enabled. Further improvement is achieved by enabling the Adapter-Event-Notification To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1853306/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
** Changed in: ubuntu-z-systems Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Bug description: The PCI Passthrough implementation is based on intercepting PCI I/O instructions which leads to a reduced I/O performance compared to execution of PCI instructions in LPAR. For improved performance the interpretive execution of the PCI store and PCI load instructions get enabled. Further improvement is achieved by enabling the Adapter-Event-Notification To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1853306/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
This bug was fixed in the package linux - 5.19.0-18.18 --- linux (5.19.0-18.18) kinetic; urgency=medium * kinetic/linux: 5.19.0-18.18 -proposed tracker (LP: #1990366) * 5.19.0-17.17: kernel NULL pointer dereference, address: 0084 (LP: #1990236) - Revert "UBUNTU: SAUCE: apparmor: Fix regression in stacking due to label flags" - Revert "UBUNTU: [Config] disable SECURITY_APPARMOR_RESTRICT_USERNS" - Revert "UBUNTU: SAUCE: Revert "hwrng: virtio - add an internal buffer"" - Revert "UBUNTU: SAUCE: Revert "hwrng: virtio - don't wait on cleanup"" - Revert "UBUNTU: SAUCE: Revert "hwrng: virtio - don't waste entropy"" - Revert "UBUNTU: SAUCE: Revert "hwrng: virtio - always add a pending request"" - Revert "UBUNTU: SAUCE: Revert "hwrng: virtio - unregister device before reset"" - Revert "UBUNTU: SAUCE: Revert "virtio-rng: make device ready before making request"" - Revert "UBUNTU: [Config] update configs after apply new apparmor patch set" - Revert "UBUNTU: SAUCE: apparmor: add user namespace creation mediation" - Revert "UBUNTU: SAUCE: selinux: Implement userns_create hook" - Revert "UBUNTU: SAUCE: bpf-lsm: Make bpf_lsm_userns_create() sleepable" - Revert "UBUNTU: SAUCE: security, lsm: Introduce security_create_user_ns()" - Revert "UBUNTU: SAUCE: lsm stacking v37: AppArmor: Remove the exclusive flag" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Add /proc attr entry for full LSM context" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Removed scaffolding function lsmcontext_init" - Revert "UBUNTU: SAUCE: lsm stacking v37: netlabel: Use a struct lsmblob in audit data" - Revert "UBUNTU: SAUCE: lsm stacking v37: Audit: Add record for multiple object contexts" - Revert "UBUNTU: SAUCE: lsm stacking v37: audit: multiple subject lsm values for netlabel" - Revert "UBUNTU: SAUCE: lsm stacking v37: Audit: Add record for multiple task security contexts" - Revert "UBUNTU: SAUCE: lsm stacking v37: Audit: Allow multiple records in an audit_buffer" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Add a function to report multiple LSMs" - Revert "UBUNTU: SAUCE: lsm stacking v37: Audit: Create audit_stamp structure" - Revert "UBUNTU: SAUCE: lsm stacking v37: Audit: Keep multiple LSM data in audit_names" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: security_secid_to_secctx module selection" - Revert "UBUNTU: SAUCE: lsm stacking v37: binder: Pass LSM identifier for confirmation" - Revert "UBUNTU: SAUCE: lsm stacking v37: NET: Store LSM netlabel data in a lsmblob" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: security_secid_to_secctx in netlink netfilter" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmcontext in security_dentry_init_security" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmcontext in security_inode_getsecctx" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmcontext in security_secid_to_secctx" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Ensure the correct LSM context releaser" - Revert "UBUNTU: SAUCE: fixup lsm stacking v37: LSM: Specify which LSM to display" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Specify which LSM to display" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmblob in security_cred_getsecid" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmblob in security_inode_getsecid" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmblob in security_current_getsecid" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmblob in security_ipc_getsecid" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmblob in security_secid_to_secctx" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmblob in security_secctx_to_secid" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmblob in security_kernel_act_as" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Use lsmblob in security_audit_rule_match" - Revert "UBUNTU: SAUCE: lsm stacking v37: IMA: avoid label collisions with stacked LSMs" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: provide lsm name and id slot mappings" - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Add the lsmblob data structure." - Revert "UBUNTU: SAUCE: lsm stacking v37: LSM: Infrastructure management of the sock security" - Revert "UBUNTU: SAUCE: lsm stacking v37: integrity: disassociate ima_filter_rule from security_audit_rule" - Revert "UBUNTU: SAUCE: apparmor: LSM stacking: switch from SK_CTX() to aa_sock()" - Revert "UBUNTU: SAUCE: apparmor: Add fine grained mediation of posix mqueues" - Revert "UBUNTU: SAUCE: apparmor: rename aa_sock() to aa_unix_sk()" -
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
This is now incl. in linux-generic | 5.19.0.18.18 | kinetic-proposed hence updating the status to Fix Committed. ** Changed in: linux (Ubuntu) Status: In Progress => Fix Committed ** Changed in: ubuntu-z-systems Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: Fix Committed Status in linux package in Ubuntu: Fix Committed Bug description: The PCI Passthrough implementation is based on intercepting PCI I/O instructions which leads to a reduced I/O performance compared to execution of PCI instructions in LPAR. For improved performance the interpretive execution of the PCI store and PCI load instructions get enabled. Further improvement is achieved by enabling the Adapter-Event-Notification To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1853306/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1853306] Re: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part
Pull request was submitted to kernel team's mailing list: https://lists.ubuntu.com/archives/kernel-team/2022-September/thread.html#133371 changing status to 'In Progress'. A test kernel was build on all major architectures in PPA and is available here: https://launchpad.net/~fheimes/+archive/ubuntu/lp1853306 ** Changed in: linux (Ubuntu) Importance: Undecided => Medium ** Changed in: linux (Ubuntu) Assignee: Frank Heimes (fheimes) => Canonical Kernel Team (canonical-kernel-team) ** Changed in: linux (Ubuntu) Status: New => In Progress ** Changed in: ubuntu-z-systems Status: New => In Progress ** Information type changed from Private to Public -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1853306 Title: [22.04 FEAT] Enhanced Interpretation for PCI Functions on s390x - kernel part Status in Ubuntu on IBM z Systems: In Progress Status in linux package in Ubuntu: In Progress Bug description: The PCI Passthrough implementation is based on intercepting PCI I/O instructions which leads to a reduced I/O performance compared to execution of PCI instructions in LPAR. For improved performance the interpretive execution of the PCI store and PCI load instructions get enabled. Further improvement is achieved by enabling the Adapter-Event-Notification To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1853306/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
