** Changed in: ubuntu-z-systems
Status: New => In Progress
** Changed in: smc-tools (Ubuntu Jammy)
Status: New => In Progress
** Changed in: smc-tools (Ubuntu Noble)
Status: New => In Progress
** Description changed:
+ SRU justification:
+
+ [ Impact ]
+
+ * IBM z17 introduces a new Network Express RDMA-capable NIC (NETD/NETH). The
+ smc_rnics tool (from smc-tools) is used to list, enable and disable SMC-R
+ capable RNICs (incl. RoCE/ISM/Network Express devices) attached to an
+ LPAR/guest, and smc_chk uses smc_rnics' output to determine the PNET ID
+ used for SMC-R/SMC-D applicability checks.
+
+ * The smc-tools versions currently in jammy (1.7.0-0ubuntu1.1) and noble
+ (1.8.3-0ubuntu1) predate upstream's support for this new PCI Function Type
+ (PFT), so:
+ - smc_rnics does not recognize/list Network Express (NETD/NETH) RNICs at
+ all (they are silently skipped from the device table), and
+ - smc_chk's PNET-ID parsing routine (which greps a fixed column out of
+ smc_rnics' output) is not aware of the two additional columns (PFT, VF)
+ that later smc_rnics versions add, so once smc_rnics is fixed, smc_chk
+ would start mis-parsing PNET IDs unless it is fixed too.
+
+ * As a result, on z17 hardware with Network Express cards passed through to
+ a guest/LPAR, smc_rnics/smc_chk fail to detect the new interface, so users
+ cannot see, enable/disable, or correctly select this RNIC for SMC-R, and
+ effectively cannot benefit from the new hardware's RDMA offload, resulting
+ in degraded network performance for SMC-R workloads.
+
+ [ Fix ]
+
+ * The fix backports five upstream smc-tools commits (listed by the IBM bug
+ reporter in comment #0) that:
+ 1) 108d5124b837 - update the smc_rnics(8) man page for --rawids/PFT/VF,
+ 2) 3cb6fbb2afac - adapt smc_chk's PNET ID parsing for the 2 new columns,
+ 3) 61129ccb308b - add the new PFT and VF columns to smc_rnics output,
+ 4) 0b965ae9b5ad - add Network Express RNIC detection/support, and
+ 5) a60b96821be3 - fix a regression (introduced by 4) where devices are not
+ detected at all when PFT is not available/known (e.g. in QEMU/KVM,
+ or for other not-yet-enlightened PCI vendor/device IDs), by falling
+ back to the older PCI vendor/device-ID based detection logic, and
+ fixing physical-port handling for multi-port RoCE Express cards.
+
+ * All five are present, unmodified, in upstream smc-tools 1.8.7/1.8.8,
+ hence resolute and stonking are not affected.
+
+ * noble's smc-tools 1.8.3 already carries all the prerequisite commits.
+
+ * jammy's smc-tools 1.7.0 is older and does not have all prerequisite
+ in place.Hence the mentioned commits do not apply cleanly.
+ To get a clean and functionally-correct result on jammy,
+ 5 additional, small prerequisite upstream commits (all already
+ shipped in newer releases) need to be applied first:
+ - smc_rnics: List unknown devices with option -a (bc0d097)
+ - smc_rnics: Display correct PNET ID for unknown Mellanox cards (d611173)
+ - smc_rnics: Include software-set PNET IDs (f74ae51)
+ - smc_chk: Indicate PNET IDs set by smc_pnet (5c589e6)
+ - smc_rnics: Recognize RoCE Express3 cards (1b4d78b)
+
+ [ Test Plan ]
+
+ * A z17 system (or LPAR/z/VM guest) with a Network Express (NETD/NETH) RNIC
+ attached/passed-through is required..
+
+ 1. Baseline (WITHOUT the fix, stock smc-tools 1.7.0-0ubuntu1.1 /
1.8.3-0ubuntu1):
+ smc_rnics
+ smc_rnics -a
+ smc_rnics -r
+ smc_chk <PID-of-an-SMC-R-using-process>
+ -> Network Express (NETD/NETH) devices are not listed at all (any PFT
+ that maps to Network Express is skipped); smc_chk's PNET-ID output
+ for other device types is unaffected (columns unchanged yet).
+
+ 2. Install the fixed packages from the PPA (ppa:fheimes/lp2165027, or
+ equivalent test PPA) into -proposed-equivalent chroot/container:
+ apt-get install -t <series>-proposed smc-tools (or from PPA)
+
+ 3. Re-run the same commands:
+ smc_rnics
+ smc_rnics -a
+ smc_rnics -r
+ smc_chk <PID-of-an-SMC-R-using-process>
+
+ * Expected results WITH the fix:
+ - smc_rnics lists Network Express (NETD/NETH) devices with correct
+ Type "Network_Express", plus the new PFT/VF columns in the "-r" raw
+ output;
+ - existing RoCE Express / RoCE Express2 / RoCE Express3 / ISM devices
+ continue to be listed exactly as before (no output regression), incl.
+ correct physical port ("PPrt") and PNET ID values;
+ - devices with PFT not available/known (e.g. under QEMU/KVM) are still
+ correctly detected via the PCI vendor/device-ID fallback path (this
+ is the specific regression a60b96821be3 fixes - verify smc_rnics
+ output is non-empty on such a system/VM, matching pre-1.8.4 behavior);
+ - smc_chk correctly extracts the PNET ID (from the new column 9) and
+ reports the same PNET ID as smc_pnet / smc_rnics for a given
+ interface/device;
+ - man page smc_rnics(8) documents the --rawids/PFT/VF semantics
+ correctly (`man smc_rnics`).
+
+ [ Where problems could occur ]
+
+ * smc_rnics/smc_chk are s390x-specific tools (SMC-R/SMC-D are s390x/RoCE/ISM
+ features), so the impact is limited to s390x users of SMC-R/SMC-D
+
+ * The tools are arch:any but the RNIC-detection code paths only ever trigger
+ on s390x PCI/sysfs layout.
+
+ * The core risk is in smc_rnics' device-type detection logic, which is now
+ more complex (PFT-based detection with a PCI-vendor/device-ID fallback,
+ plus a new "multiport" flag replacing an ad-hoc pft-value check for
+ deciding whether to read the per-interface "dev_port" sysfs attribute).
+ A mistake here could cause:
+ - existing RoCE Express / RoCE Express2 / RoCE Express3 / ISM devices to
+ disappear from smc_rnics output, or show wrong Type/Port/PNET-ID values;
+ - smc_chk to derive a wrong PNET ID for existing device types, which could
+ make SMC-R/SMC-D silently not be used (performance regression only)
+
+ * On jammy specifically, 5 add. (non-z17) upstream commits had to
+ be backported first as prerequisites.
+ While all of this is already shipped unmodified in noble (and later)
+ for >1 year, it increases the diff size/review surface for the jammy SRU
+ compared to the 5 requested patches only.
+ This is a deliberate trade-off: the alternative (attempting to
+ force/backport the 5 bug-listed commits onto 1.7.0 without their prereq)
+ is not appropriate and clean and would silently reference nonexistent
+ shell functions/variables (a de-facto broken package),
+ which is a worse outcome and would likely be rejected anyway.
+
+ * Mitigation:
+ - all 5 (noble) / 10 (jammy) patches are individual, minimal, upstream
+ commits carrying full DEP-3 headers referencing the exact upstream
+ commit and this LP bug, so each hunk is traceable and independently
+ revertable/bisectable;
+ - after patching, jammy's and noble's smc_rnics/smc_chk/smc_rnics.8 are
+ byte-for-byte identical (aside from VERSION string/copyright year),
+ i.e. the same code path is exercised and tested on both series;
+ - changes are confined to smc_rnics (bash script), smc_chk (bash script)
+ and their man page; no changes to the LD_PRELOAD library, build system,
+ or any other binary/compiled component;
+ - the package builds and installs cleanly, patches apply without fuzz,
+ and lintian shows no new errors/warnings
+ - build has been validated in PPA for all architectures
+ https://launchpad.net/~fheimes/+archive/ubuntu/lp2165027
+
+ __________
+
Problem description:
--------------------
"New cards/functionality of the IBM z17 (NETD & NETH) will not be
displayed correctly, so customers may not use the technology in a
correct way resulting in low performance and dissatisfaction with the
platform and / or os".
Solution:
---------
- The following list of backports is required to support Network Express cards
on z17 correctly.
+ The following list of backports is required to support Network Express cards
on z17 correctly.
All fixes are included in smc-tools v1.8.7 and above, therefore, the
following 5 patches need to be applied to Jammy and Noble in the listed order:
1) 108d5124b837 man pages: Update man page for --rawids option and PFT and VF
columns
2) 3cb6fbb2afac smc_rnics: update smc_chk to extract PNetID from column 9
3) 61129ccb308b smc_rnics: Add PFT and VF columns to smc_rnics output
4) 0b965ae9b5ad smc_rnics: Add support for Network Express RNIC in smc_rnics
5) a60b96821be3 smc_rnics: fix regression when PFT not available
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2165027
Title:
[UBUNTU 22.04] smc_rnics is not listing the NetH/NetD cards inside the
guest
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/2165027/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
