** Description changed:
+ [Impact]
+
+ * Users of a NetApp E-Series RDAC due to very long names increasing
+ the VDP page and and cause multipathd to hang
+
+ * Introduced as side effect of other changes in 7.4 and fixed later this
+ is backporting the HW exploitation to read an enhanced VDP in this case
+ which in such cases works fine again.
+
+ [Test Case]
+
+ * #1 Case: Test on a NetApp E-Series RDAC, the reporter has such HW and
+ is able to do so.
+
+ * #2 regression tests: since this touches multipath discovery any such
+ cases would be interesting tests. From installers using multipath on
+ mainframes to multipath-iscsi disks on an raspi. The Range is huge and
+ unclear how much exactly should be done. I can myself do said mainframe
+ checks and utilize the pkgtests to run through iscsid.
+ In addition we should keep this one a bit longer in -proposed IMHO.
+
+ [Regression Potential]
+
+ * All cases asking for vpd 0x83 should be unchanged and only those asking
+ for an 0xc9 should get the new improved behavior. But I'm not a 100%
+ sure on potential side effects yet. Changes and thereby regressions
+ would be limited to the device discovery of multipathd, so that is what
+ one needs to take an extra eye on e.g. pre/post list of multipath
+ devices (and attributes as those are populated via vpd).
+
+ [Other Info]
+
+ * n/a
+
+ ---
+
I've notice recently that multipathd started hanging when trying to
handle long iscsi target names. In my case these targets are generated
by the OpenStack cinder driver for SolidFire.
My guess is that the problem is occurring due to a bug introduced in
multipath-tools 0.7.4, by commit "limbultipath: prefer RDAC checker with
detect_checker*"*[1].
This change adds a routine to check RDAC compatibility during the
detect_checker procedure, which runs into an infinite loop in the piece
of code that tries to gather SCSI VPD pages larger than 254 bytes (this
is my case due to long iSCSI Target names).
This issue was later addressed by several commits in 0.7.5 and 0.7.6:
"libmultipath: sgio_get_vpd: add page argument" [2], "libmultipath: Fix
sgio_get_vpd()" [3], "libmultipath: fix return code of sgio_get_vpd()"
[4], "libmultipath: get_vpd_sgio: support VPD 0xc9" [5].
I've found a temporary work around for this by using select_checker =
“no” in Ubuntu 18. This appears to avoid the problem by skipping RDAC
compatibility check code.
Also, I've also tested 0.8.3 in a RH environment and found no issues
running default select_checker = “yes” (Ubuntu 20 also delivers 0.8.3).
-
So, in short:
- - Looks like Ubuntu multipath-tools is broken,
- - There is a fix in newer versions that aren’t in the ubuntu 18 repo yet.
+ - Looks like Ubuntu multipath-tools is broken,
+ - There is a fix in newer versions that aren’t in the ubuntu 18 repo yet.
- The OpenStack Cinder driver for Solidfire just happen to hit this bug
because of the long names
- Setting select_checker = “no” is an possible work-around until Ubuntu
publishes 0.7.6
Bellow "multipath -r -v3" output from my env:
:/$ cat /etc/multipath.conf
- devices {
- device {
- vendor "SolidFir" product "SSD SAN" detect_checker "no"
- }
+ devices {
+ device {
+ vendor "SolidFir" product "SSD SAN" detect_checker "no"
+ }
}
:/$ sudo multipath -r -v3 Jul 09 19:06:24 | loading
//lib/multipath/libchecktur.so checker Jul 09 19:06:24 | loading
//lib/multipath/libprioconst.so prioritizer Jul 09 19:06:24 | vda:
blacklisted, udev property missing Jul 09 19:06:24 | sda: udev property
ID_WWN whitelisted Jul 09 19:06:24 | sda: mask = 0x1f Jul 09 19:06:24 |
sda: dev_t = 8:0 Jul 09 19:06:24 | sda: size = 10485760 Jul 09 19:06:24
| sda: vendor = SolidFir Jul 09 19:06:24 | sda: product = SSD SAN Jul 09
19:06:24 | sda: rev = 0001 Jul 09 19:06:24 | sda: h:b:t:l = 2:0:0:0 Jul
09 19:06:24 | sda: tgt_node_name = iqn.2010-01.com.solidfire:xpsi.uuid-
7b8a927e-4dbc-4a2b-b47a-1bde324895eb.53 Jul 09 19:06:24 | sda: path
state = running Jul 09 19:06:24 | sda: 1018 cyl, 166 heads, 62
sectors/track, start at 0 Jul 09 19:06:24 | sda: serial =
7870736900000035f47acc0100000000 Jul 09 19:06:24 | sda: get_state Jul 09
19:06:24 | sda: detect_checker = no (setting: storage device
configuration) Jul 09 19:06:24 | sda: path_checker = tur (setting:
multipath internal) Jul 09 19:06:24 | sda: checker timeout = 30 s
(setting: multipath internal) Jul 09 19:06:24 | sda: tur state = up Jul
09 19:06:24 | sda: uid_attribute = ID_SERIAL (setting: multipath
internal) Jul 09 19:06:24 | sda: uid = 36f47acc1000000007870736900000035
(udev) Jul 09 19:06:24 | sda: detect_prio = yes (setting: multipath
internal) Jul 09 19:06:24 | sda: prio = const (setting: multipath
internal) Jul 09 19:06:24 | sda: prio args = "" (setting: multipath
internal) Jul 09 19:06:24 | sda: const prio = 1 Jul 09 19:06:24 | loop0:
blacklisted, udev property missing Jul 09 19:06:24 | loop1: blacklisted,
udev property missing Jul 09 19:06:24 | loop2: blacklisted, udev
property missing Jul 09 19:06:24 | loop3: blacklisted, udev property
missing Jul 09 19:06:24 | loop4: blacklisted, udev property missing Jul
09 19:06:24 | loop5: blacklisted, udev property missing Jul 09 19:06:24
| loop6: blacklisted, udev property missing Jul 09 19:06:24 | loop7:
blacklisted, udev property missing ===== paths list ===== uuid hcil dev
dev_t pri dm_st chk_st vend/prod 36f47acc1000000007870736900000035
2:0:0:0 sda 8:0 1 undef undef SolidFir, Jul 09 19:06:24 | libdevmapper
version 1.02.145 (2017-11-03) Jul 09 19:06:24 | DM multipath kernel
driver v1.13.0 Jul 09 19:06:24 | sda: udev property ID_WWN whitelisted
Jul 09 19:06:24 | sda: (SolidFir:SSD SAN) vendor/product blacklisted Jul
09 19:06:24 | sda: orphan path, wwid blacklisted Jul 09 19:06:24 | const
prioritizer refcount 1 Jul 09 19:06:24 | tur checker refcount 1 Jul 09
19:06:24 | unloading const prioritizer Jul 09 19:06:24 | unloading tur
checker
[1]https://git.opensvc.com/gitweb.cgi?p=multipath-
tools/.git;a=commit;h=84e16fbdb6648e685b65781d6ef0742469bd4b18
[2]https://git.opensvc.com/gitweb.cgi?p=multipath-
tools/.git;a=commit;h=d74a79f3590f3f28d34287b64c565fd5bf9705d0
[3]https://git.opensvc.com/gitweb.cgi?p=multipath-
tools/.git;a=commit;h=1f401b69e478b35d4b54b1980be67943c6d61d9f
[4]https://git.opensvc.com/gitweb.cgi?p=multipath-
tools/.git;a=commit;h=c6140952d1b682261c07cdb7138c944ee83d8809
[5]https://git.opensvc.com/gitweb.cgi?p=multipath-
tools/.git;a=commit;h=9d0e6bffbb3e490e92cf91444cfeb4177d540707
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1891202
Title:
Multipathd hangs with long iscsi target names in Ubuntu 18.04
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1891202/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
