[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
** Changed in: ubuntu-z-systems Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Focal: Fix Released Status in linux source package in Groovy: Fix Released Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
This bug was fixed in the package linux - 5.4.0-42.46 --- linux (5.4.0-42.46) focal; urgency=medium * focal/linux: 5.4.0-42.46 -proposed tracker (LP: #1887069) * linux 4.15.0-109-generic network DoS regression vs -108 (LP: #1886668) - SAUCE: Revert "netprio_cgroup: Fix unlimited memory leak of v2 cgroups" linux (5.4.0-41.45) focal; urgency=medium * focal/linux: 5.4.0-41.45 -proposed tracker (LP: #1885855) * Packaging resync (LP: #1786013) - update dkms package versions * CVE-2019-19642 - kernel/relay.c: handle alloc_percpu returning NULL in relay_open * CVE-2019-16089 - SAUCE: nbd_genl_status: null check for nla_nest_start * CVE-2020-11935 - aufs: do not call i_readcount_inc() * ip_defrag.sh in net from ubuntu_kernel_selftests failed with 5.0 / 5.3 / 5.4 kernel (LP: #1826848) - selftests: net: ip_defrag: ignore EPERM * Update lockdown patches (LP: #1884159) - SAUCE: acpi: disallow loading configfs acpi tables when locked down * seccomp_bpf fails on powerpc (LP: #1885757) - SAUCE: selftests/seccomp: fix ptrace tests on powerpc * Introduce the new NVIDIA 418-server and 440-server series, and update the current NVIDIA drivers (LP: #1881137) - [packaging] add signed modules for the 418-server and the 440-server flavours -- Khalid Elmously Thu, 09 Jul 2020 19:50:26 -0400 ** Changed in: linux (Ubuntu Groovy) Status: In Progress => Fix Released ** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2019-16089 ** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2019-19642 ** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-11935 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Fix Committed Status in linux package in Ubuntu: Fix Released Status in linux source package in Focal: Fix Released Status in linux source package in Groovy: Fix Released Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
** Also affects: linux (Ubuntu Groovy) Importance: Undecided Assignee: Canonical Kernel Team (canonical-kernel-team) Status: In Progress -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Fix Committed Status in linux package in Ubuntu: In Progress Status in linux source package in Focal: Fix Released Status in linux source package in Groovy: In Progress Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
This bug was fixed in the package linux - 5.4.0-40.44 --- linux (5.4.0-40.44) focal; urgency=medium * linux-oem-5.6-tools-common and -tools-host should be dropped (LP: #1881120) - [Packaging] Add Conflicts/Replaces to remove linux-oem-5.6-tools-common and -tools-host * Packaging resync (LP: #1786013) - [Packaging] update helper scripts * Slow send speed with Intel I219-V on Ubuntu 18.04.1 (LP: #1802691) - e1000e: Disable TSO for buffer overrun workaround * CVE-2020-0543 - UBUNTU/SAUCE: x86/speculation/srbds: do not try to turn mitigation off when not supported * Realtek 8723DE [10ec:d723] subsystem [10ec:d738] disconnects unsolicitedly when Bluetooth is paired: Reason: 23=IEEE8021X_FAILED (LP: #1878147) - SAUCE: Revert "UBUNTU: SAUCE: rtw88: Move driver IQK to set channel before association for 11N chip" - SAUCE: Revert "UBUNTU: SAUCE: rtw88: fix rate for a while after being connected" - SAUCE: Revert "UBUNTU: SAUCE: rtw88: No retry and report for auth and assoc" - SAUCE: Revert "UBUNTU: SAUCE: rtw88: 8723d: Add coex support" - rtw88: add a debugfs entry to dump coex's info - rtw88: add a debugfs entry to enable/disable coex mechanism - rtw88: 8723d: Add coex support - SAUCE: rtw88: coex: 8723d: set antanna control owner - SAUCE: rtw88: coex: 8723d: handle BT inquiry cases - SAUCE: rtw88: fix EAPOL 4-way failure by finish IQK earlier * CPU stress test fails with focal kernel (LP: #1867900) - [Config] Disable hisi_sec2 temporarily * Enforce all config annotations (LP: #1879327) - [Config]: do not enforce CONFIG_VERSION_SIGNATURE - [Config]: prepare to enforce all - [Config]: enforce all config options * Focal update: v5.4.44 upstream stable release (LP: #1881927) - ax25: fix setsockopt(SO_BINDTODEVICE) - dpaa_eth: fix usage as DSA master, try 3 - net: don't return invalid table id error when we fall back to PF_UNSPEC - net: dsa: mt7530: fix roaming from DSA user ports - net: ethernet: ti: cpsw: fix ASSERT_RTNL() warning during suspend - __netif_receive_skb_core: pass skb by reference - net: inet_csk: Fix so_reuseport bind-address cache in tb->fast* - net: ipip: fix wrong address family in init error path - net/mlx5: Add command entry handling completion - net: mvpp2: fix RX hashing for non-10G ports - net: nlmsg_cancel() if put fails for nhmsg - net: qrtr: Fix passing invalid reference to qrtr_local_enqueue() - net: revert "net: get rid of an signed integer overflow in ip_idents_reserve()" - net sched: fix reporting the first-time use timestamp - net/tls: fix race condition causing kernel panic - nexthop: Fix attribute checking for groups - r8152: support additional Microsoft Surface Ethernet Adapter variant - sctp: Don't add the shutdown timer if its already been added - sctp: Start shutdown on association restart if in SHUTDOWN-SENT state and socket is closed - tipc: block BH before using dst_cache - net/mlx5e: kTLS, Destroy key object after destroying the TIS - net/mlx5e: Fix inner tirs handling - net/mlx5: Fix memory leak in mlx5_events_init - net/mlx5e: Update netdev txq on completions during closure - net/mlx5: Fix error flow in case of function_setup failure - net/mlx5: Annotate mutex destroy for root ns - net/tls: fix encryption error checking - net/tls: free record only on encryption error - net: sun: fix missing release regions in cas_init_one(). - net/mlx4_core: fix a memory leak bug. - mlxsw: spectrum: Fix use-after-free of split/unsplit/type_set in case reload fails - ARM: dts: rockchip: fix phy nodename for rk3228-evb - ARM: dts: rockchip: fix phy nodename for rk3229-xms6 - arm64: dts: rockchip: fix status for in rk3328-evb.dts - arm64: dts: rockchip: swap interrupts interrupt-names rk3399 gpu node - ARM: dts: rockchip: swap clock-names of gpu nodes - ARM: dts: rockchip: fix pinctrl sub nodename for spi in rk322x.dtsi - gpio: tegra: mask GPIO IRQs during IRQ shutdown - ALSA: usb-audio: add mapping for ASRock TRX40 Creator - net: microchip: encx24j600: add missed kthread_stop - gfs2: move privileged user check to gfs2_quota_lock_check - gfs2: Grab glock reference sooner in gfs2_add_revoke - drm/amdgpu: drop unnecessary cancel_delayed_work_sync on PG ungate - drm/amd/powerplay: perform PG ungate prior to CG ungate - drm/amdgpu: Use GEM obj reference for KFD BOs - cachefiles: Fix race between read_waiter and read_copier involving op->to_do - usb: dwc3: pci: Enable extcon driver for Intel Merrifield - usb: phy: twl6030-usb: Fix a resource leak in an error handling path in 'twl6030_usb_probe()' - usb: gadget: legacy: fix redundant initialization warnings - net: freescale: select CONFIG_FIXED_PHY where needed - IB/i40iw: Remove bogus
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
Many thanks! Adjusting tags. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Fix Committed Status in linux package in Ubuntu: In Progress Status in linux source package in Focal: Fix Committed Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1 VF 1 :00:00.7 PCHID 0100 PF 1 VF 2
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
Many thanks! Adjust tags. ** Tags removed: verification-needed-focal ** Tags added: verification-done-focal -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Fix Committed Status in linux package in Ubuntu: In Progress Status in linux source package in Focal: Fix Committed Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-focal -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Fix Committed Status in linux package in Ubuntu: In Progress Status in linux source package in Focal: Fix Committed Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
** Changed in: ubuntu-z-systems Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Fix Committed Status in linux package in Ubuntu: In Progress Status in linux source package in Focal: Fix Committed Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
** Changed in: linux (Ubuntu Focal) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: In Progress Status in linux package in Ubuntu: In Progress Status in linux source package in Focal: Fix Committed Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
** Also affects: linux (Ubuntu Focal) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Focal) Importance: Undecided => Medium ** Changed in: linux (Ubuntu Focal) Status: New => In Progress -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: In Progress Status in linux package in Ubuntu: In Progress Status in linux source package in Focal: In Progress Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
First of all thanks for testing the PPA. I'm sure that helps in regard to the acceptance of the SRU. For the additional two patches it's best to open a separate LP ticket - and upstream acceptance is of course crucial, too. The common code change is quite critical (is it one of the two - or a third one?). I assume that you tried to make the needed modifications as small and limited as possible, clear and traceable. Since the these patches are based on one ones from above, I guess it's not possible to flag them upstream for an upstream release update - for 5.4 (at least the common code change)?! Please let me know the commit title and/or IDs. We have to have a deep look into it and a solid SRU justification -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: In Progress Status in linux package in Ubuntu: In Progress Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
@IBM Please can you test the patched kernel from the PPA and leave a quick feedback here if it works like expected? This would lower the risk a bit and increase the confidence and would at the end help regarding the acceptance by the kernel team. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: In Progress Status in linux package in Ubuntu: In Progress Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
A test build is available at 'ppa:fheimes/tmp-s390x'. ( For more info see https://launchpad.net/~fheimes/+archive/ubuntu/tmp-s390x ) The kernel and supporting packages for this ticket are under "linux - 5.4.0-31.35ubuntu1" -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: In Progress Status in linux package in Ubuntu: In Progress Bug description: SRU Justification: == [Impact] * Mellanox CX5 port multi-pathing is broken on s390x due to non- standard topology of PCI IDs (phys. and virtual) * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that can be used to combine multiple networking ports to improve performance and reliability. * For that purpose, the mlx5 driver combines PCI functions based on topology information (the function number) as determined by their PCI ID. * Currently the Linux on Z PCI bus does not reflect PCI topology information in the PCI ID. As a result, the mlx5 multi-path function is broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
Based on comment #28 the bug description / SRU justification was updated. ** Description changed: SRU Justification: == [Impact] - * On s390x the enumeration of PCI functions does not reflect which - functions belongs to which physical adapter. + * Mellanox CX5 port multi-pathing is broken on s390x due to non-standard + topology of PCI IDs (phys. and virtual) - * Layout of a PCI function address on Linux: - :00:00.0 - ::. + * The Mellanox Connect-X 5 PCI driver (mlx5) implements multi-path that + can be used to combine multiple networking ports to improve performance + and reliability. - * On s390x, each function is presented as individual root complex today, e.g.: - PCHID 0100 VF1 :00:00.0 - PCHID 0100 VF23 0001:00:00.0 - PCHID 0200 VF1 0002:00:00.0 - OCHID 0100 VF17 0003:00:00.0 + * For that purpose, the mlx5 driver combines PCI functions based on + topology information (the function number) as determined by their PCI + ID. - * On other platforms, the addresses correctly reflect the actual HW - configuration. - - * Some device drivers (like mlx5, for Mellanox adapters) group functions - of one physical adapter by checking which PCI functions have identical - values for ::. - - * We need to use the same enumeration scheme to achieve this - functionality on s390x. - - * In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, - and all virtual functions need to use the same : numbers with function/device numbers counting up. - - * Required result (example with 4 VFs per PF): - PCHID 0100 PF 0 :00:00.0 - PCHID 0100 PF 1 :00:00.1 - PCHID 0100 PF 0 VF 0 :00:00.2 - PCHID 0100 PF 0 VF 1 :00:00.3 - PCHID 0100 PF 0 VF 2 :00:00.4 - PCHID 0100 PF 0 VF 3 :00:00.5 - PCHID 0100 PF 1 VF 0 :00:00.6 - PCHID 0100 PF 1 VF 1 :00:00.7 - PCHID 0100 PF 1 VF 2 :00:00.8 - PCHID 0100 PF 1 VF 3 :00:00.9 - PCHID 0200 PF 0 0001:00:00.0 + * Currently the Linux on Z PCI bus does not reflect PCI topology + information in the PCI ID. As a result, the mlx5 multi-path function is + broken and cannot be activated. [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __ Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
Kernel SRU request submitted: https://lists.ubuntu.com/archives/kernel-team/2020-May/thread.html#110043 Updating status to 'In Progress'. ** Description changed: + SRU Justification: + == + + [Impact] + + * On s390x the enumeration of PCI functions does not reflect which + functions belongs to which physical adapter. + + * Layout of a PCI function address on Linux: + :00:00.0 + ::. + + * On s390x, each function is presented as individual root complex today, e.g.: + PCHID 0100 VF1 :00:00.0 + PCHID 0100 VF23 0001:00:00.0 + PCHID 0200 VF1 0002:00:00.0 + OCHID 0100 VF17 0003:00:00.0 + + * On other platforms, the addresses correctly reflect the actual HW + configuration. + + * Some device drivers (like mlx5, for Mellanox adapters) group functions + of one physical adapter by checking which PCI functions have identical + values for ::. + + * We need to use the same enumeration scheme to achieve this + functionality on s390x. + + * In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, + and all virtual functions need to use the same : numbers with function/device numbers counting up. + + * Required result (example with 4 VFs per PF): + PCHID 0100 PF 0 :00:00.0 + PCHID 0100 PF 1 :00:00.1 + PCHID 0100 PF 0 VF 0 :00:00.2 + PCHID 0100 PF 0 VF 1 :00:00.3 + PCHID 0100 PF 0 VF 2 :00:00.4 + PCHID 0100 PF 0 VF 3 :00:00.5 + PCHID 0100 PF 1 VF 0 :00:00.6 + PCHID 0100 PF 1 VF 1 :00:00.7 + PCHID 0100 PF 1 VF 2 :00:00.8 + PCHID 0100 PF 1 VF 3 :00:00.9 + PCHID 0200 PF 0 0001:00:00.0 + + [Fix] + + * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- + Improve-handling-of-unset-UID.patch + + * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- + embedding-hotplug_slot-in-zdev.patch + + * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- + Expose-new-port-attribute-for-PCIe-function.patch + + * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- + adaptation-of-iommu-to-multifunction.patch + + * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- + define-kernel-parameters-for-PCI-multifunct.patch + + * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- + define-RID-and-RID-available.patch + + * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- + create-zPCI-bus.patch + + * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- + adapt-events-for-zbus.patch + + * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- + Handling-multifunctions.patch + + * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- + Do-not-disable-PF-when-VFs-exist.patch + + * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- + Documentation-for-zPCI.patch + + * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- + removes-wrong-PCI-multifunction-assignment.patch + + [Test Case] + + * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more + RoCE Express PCI 2(.1) adapters. + + * Assign the adapters (and it's virtual functions) to an LPAR. + + * Verify whether the physical and virtual functions are grouped in + arbitrary order or in consecutive order - physical first (for example + with lspci -t ...) + + [Regression Potential] + + * The regression potential can be considered as moderate, since: + + * It is purely s390x specific code (arch/s390/* + drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and + some doc adjustments, too). + + * It largely affects zPCI, the s390x specific PCI code layer. + + * PCI cards available for s390x are optional cards (RoCE and zEDC) and + not very wide-spread. + + * The situation described above affects the RoCE adapters only (Mellanox + based). + + * The patches are also upstream accepted and available via linux-next, + but to apply them to focal kernel 5.4 the above backports are needed. + + * However, the code is modified by several patches (12), hence there is + a chance to break zPCI with them. + + * For upfront testing a PPA got created with a focal (master-next) + kernel that incl. all the above patches. + + __ + Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
The backports seem to apply, compile and build fine. The only little issue is that the backports do not come with full provenance, which is important for the Canonical kernel team. Means that the 'Signed-off-by\|Reviewed-by\|Tested-by' lines from the upstream accepted commits need to be added to the corresponding backports, as well as a backport lines that includes the upstream commit id. Here, especially the backport lines were all missing. I've looked them up and added them now by hand, but would like to ask you to directly add them to the backports next time. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: In Progress Status in linux package in Ubuntu: In Progress Bug description: SRU Justification: == [Impact] * On s390x the enumeration of PCI functions does not reflect which functions belongs to which physical adapter. * Layout of a PCI function address on Linux: :00:00.0 ::. * On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 * On other platforms, the addresses correctly reflect the actual HW configuration. * Some device drivers (like mlx5, for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. * We need to use the same enumeration scheme to achieve this functionality on s390x. * In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. * Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1 VF 1 :00:00.7 PCHID 0100 PF 1 VF 2 :00:00.8 PCHID 0100 PF 1 VF 3 :00:00.9 PCHID 0200 PF 0 0001:00:00.0 [Fix] * Backport 1: https://launchpadlibrarian.net/479699471/0001-s390-pci- Improve-handling-of-unset-UID.patch * Backport 2: https://launchpadlibrarian.net/479699482/0002-s390-pci- embedding-hotplug_slot-in-zdev.patch * Backport 3: https://launchpadlibrarian.net/479699492/0003-s390-pci- Expose-new-port-attribute-for-PCIe-function.patch * Backport 4: https://launchpadlibrarian.net/479699497/0004-s390-pci- adaptation-of-iommu-to-multifunction.patch * Backport 5: https://launchpadlibrarian.net/479700706/0005-s390-pci- define-kernel-parameters-for-PCI-multifunct.patch * Backport 6: https://launchpadlibrarian.net/479700712/0006-s390-pci- define-RID-and-RID-available.patch * Backport 7: https://launchpadlibrarian.net/479700739/0007-s390-pci- create-zPCI-bus.patch * Backport 8: https://launchpadlibrarian.net/479700769/0008-s390-pci- adapt-events-for-zbus.patch * Backport 9: https://launchpadlibrarian.net/479700786/0009-s390-pci- Handling-multifunctions.patch * Backport 10: https://launchpadlibrarian.net/479700794/0010-s390-pci- Do-not-disable-PF-when-VFs-exist.patch * Backport 11: https://launchpadlibrarian.net/479700798/0011-s390-pci- Documentation-for-zPCI.patch * Backport 12: https://launchpadlibrarian.net/479700799/0012-s390-pci- removes-wrong-PCI-multifunction-assignment.patch [Test Case] * Prepare an IBM z13 or LinuxONE III (or newer) system with two or more RoCE Express PCI 2(.1) adapters. * Assign the adapters (and it's virtual functions) to an LPAR. * Verify whether the physical and virtual functions are grouped in arbitrary order or in consecutive order - physical first (for example with lspci -t ...) [Regression Potential] * The regression potential can be considered as moderate, since: * It is purely s390x specific code (arch/s390/* drivers/iommu/s390-iommu.c and drivers/pci/hotplug/s390_pci_hpc.c - and some doc adjustments, too). * It largely affects zPCI, the s390x specific PCI code layer. * PCI cards available for s390x are optional cards (RoCE and zEDC) and not very wide-spread. * The situation described above affects the RoCE adapters only (Mellanox based). * The patches are also upstream accepted and available via linux-next, but to apply them to focal kernel 5.4 the above backports are needed. * However, the code is modified by several patches (12), hence there is a chance to break zPCI with them. * For upfront testing a PPA got created with a focal (master-next) kernel that incl. all the above patches. __
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
Sounds good - I tried to cherry pick the commits once, but it failed - but I don't remember why. Could be of course due to the power mgnt removal ... (in comment #5 I just count 10 commits - and you mentioned 13) In a call earlier today this was briefly discusses with Nicholas and Heinz-Werner. My assumption was that this is mainly for the gt kernel, means could be applied to that only. But it turned out that this also requested/needed for the stock ubuntu-server kernel. I hope there is on interference with the other pci related tickets we worked on. They will land in the focal master-next tree soon - so probably best to double check the backports here on the updated tree. Then we can only follow the SRU process. Hence, I suggest that you please attach the backports (that as far as I could see are all architecture specific) - or mention if any of the commits are cherry-pick-able - and I'll apply, compile and submit them as SRU to the kernel teams mailing list and we have to discuss with the kernel team what's possible/acceptable. Means business as usual (like with the other tickets, too). ** Changed in: ubuntu-z-systems Assignee: (unassigned) => Skipper Bug Screeners (skipper-screen-team) ** Changed in: linux (Ubuntu) Assignee: Skipper Bug Screeners (skipper-screen-team) => Canonical Kernel Team (canonical-kernel-team) ** Changed in: linux (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Incomplete Status in linux package in Ubuntu: Incomplete Bug description: Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1 VF 1 :00:00.7 PCHID 0100 PF 1 VF 2 :00:00.8 PCHID 0100 PF 1 VF 3 :00:00.9 PCHID 0200 PF 0 0001:00:00.0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1874056/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
Compiles now on 18.04 with 2nd backport instead of the cherry-pick. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Incomplete Status in linux package in Ubuntu: New Bug description: Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1 VF 1 :00:00.7 PCHID 0100 PF 1 VF 2 :00:00.8 PCHID 0100 PF 1 VF 3 :00:00.9 PCHID 0200 PF 0 0001:00:00.0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1874056/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
The commits landed in between in linux-next (not tagged yet, but that's fine). So it's all brand new stuff ... I think the chance it not very high that they are just cleanly cherry-pick-able - having the other PCI changes in mind, too. I guess you are thinking about backporting them to focal master-next? $ git clone https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal --branch master-next --single-branch focal-master-next -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Incomplete Status in linux package in Ubuntu: New Bug description: Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1 VF 1 :00:00.7 PCHID 0100 PF 1 VF 2 :00:00.8 PCHID 0100 PF 1 VF 3 :00:00.9 PCHID 0200 PF 0 0001:00:00.0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1874056/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
According to the bug title this all should still land in 20.04? Please notice that 20.04 got already released, hence the development is over and the window to add new features is largely closed. Requesting changes to an already released Ubuntu version is now strictly regulated by the Stable Release Upgrade (SRU) process: https://wiki.ubuntu.com/StableReleaseUpdates Especially: https://wiki.ubuntu.com/StableReleaseUpdates#When shows that the SRU process is mainly for critical bugs, security issues and regressions. Based on these SRU guidelines, the above patches now need to be strictly screened. For larger patch sets there is usually the HWE kernel available to get things in - or (especially when reading about Cloud) a custom kernel. Such larger modifications and updates should generally land when the target Ubuntu release is still in development. Next step is to have a look at the code and where it currently is (upstream) ... -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Incomplete Status in linux package in Ubuntu: New Bug description: Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1 VF 1 :00:00.7 PCHID 0100 PF 1 VF 2 :00:00.8 PCHID 0100 PF 1 VF 3 :00:00.9 PCHID 0200 PF 0 0001:00:00.0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1874056/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
The mentioned modification makes sense. But was this change already brought upstream? And if so you you please share the commit (or backport) that does apply cleanly to the focal master-next tree? ** Changed in: ubuntu-z-systems Status: New => Incomplete ** Changed in: ubuntu-z-systems Importance: Undecided => Medium -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: Incomplete Status in linux package in Ubuntu: New Bug description: Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1 VF 1 :00:00.7 PCHID 0100 PF 1 VF 2 :00:00.8 PCHID 0100 PF 1 VF 3 :00:00.9 PCHID 0200 PF 0 0001:00:00.0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1874056/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
** Description changed: - Description will follow + Today, the enumeration of PCI functions on s390x does not reflect which + functions belongs to which physical adapter. + + Layout of a PCI function address on Linux: + :00:00.0 + ::. + + On s390x, each function is presented as individual root complex today, + e.g.: + + PCHID 0100 VF1 :00:00.0 + PCHID 0100 VF23 0001:00:00.0 + PCHID 0200 VF1 0002:00:00.0 + OCHID 0100 VF17 0003:00:00.0 + + On other platforms, the addresses correctly reflect the actual HW + configuration. Some device drivers (mlx5 for Mellanox adapters) group + functions of one physical adapter by checking which PCI functions have + identical values for ::. We need to use the + same enumeration scheme to achieve this functionality on s390x. + + In this case, the two physical functions of a Mellanox adapter need to + get function number 0 and 1, and all virtual functions need to use the + same : numbers with function/device numbers counting + up. + + Required result (example with 4 VFs per PF): + + PCHID 0100 PF 0 :00:00.0 + PCHID 0100 PF 1 :00:00.1 + PCHID 0100 PF 0 VF 0 :00:00.2 + PCHID 0100 PF 0 VF 1 :00:00.3 + PCHID 0100 PF 0 VF 2 :00:00.4 + PCHID 0100 PF 0 VF 3 :00:00.5 + PCHID 0100 PF 1 VF 0 :00:00.6 + PCHID 0100 PF 1 VF 1 :00:00.7 + PCHID 0100 PF 1 VF 2 :00:00.8 + PCHID 0100 PF 1 VF 3 :00:00.9 + + PCHID 0200 PF 0 0001:00:00.0 ** Also affects: ubuntu-z-systems Importance: Undecided Status: New -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in Ubuntu on IBM z Systems: New Status in linux package in Ubuntu: New Bug description: Today, the enumeration of PCI functions on s390x does not reflect which functions belongs to which physical adapter. Layout of a PCI function address on Linux: :00:00.0 ::. On s390x, each function is presented as individual root complex today, e.g.: PCHID 0100 VF1 :00:00.0 PCHID 0100 VF23 0001:00:00.0 PCHID 0200 VF1 0002:00:00.0 OCHID 0100 VF17 0003:00:00.0 On other platforms, the addresses correctly reflect the actual HW configuration. Some device drivers (mlx5 for Mellanox adapters) group functions of one physical adapter by checking which PCI functions have identical values for ::. We need to use the same enumeration scheme to achieve this functionality on s390x. In this case, the two physical functions of a Mellanox adapter need to get function number 0 and 1, and all virtual functions need to use the same : numbers with function/device numbers counting up. Required result (example with 4 VFs per PF): PCHID 0100 PF 0 :00:00.0 PCHID 0100 PF 1 :00:00.1 PCHID 0100 PF 0 VF 0 :00:00.2 PCHID 0100 PF 0 VF 1 :00:00.3 PCHID 0100 PF 0 VF 2 :00:00.4 PCHID 0100 PF 0 VF 3 :00:00.5 PCHID 0100 PF 1 VF 0 :00:00.6 PCHID 0100 PF 1 VF 1 :00:00.7 PCHID 0100 PF 1 VF 2 :00:00.8 PCHID 0100 PF 1 VF 3 :00:00.9 PCHID 0200 PF 0 0001:00:00.0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1874056/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1874056] Re: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter
** Summary changed: - s390x/pci: enumerate pci functions per physical adapter + [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1874056 Title: [UBUNTU 20.04] s390x/pci: enumerate pci functions per physical adapter Status in linux package in Ubuntu: New Bug description: Description will follow To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1874056/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
