I verified this works correctly on 2 LPARs, including the original LPAR from
the bug report. (Thanks, Michail)
Marking as verification-done.
Installed the version from -proposed:
root@pinelp2:~# dpkg -l | grep 0.4.9-3ubuntu7.8
ii kpartx 0.4.9-3ubuntu7.8
ppc64el create device mappings for partitions
ii kpartx-boot 0.4.9-3ubuntu7.8
all Provides kpartx during boot
ii multipath-tools 0.4.9-3ubuntu7.8
ppc64el maintain multipath block device access
ii multipath-tools-boot 0.4.9-3ubuntu7.8
all Support booting from multipath devices
root@pinelp3:~# dpkg -l | grep 0.4.9-3ubuntu7.8
ii kpartx 0.4.9-3ubuntu7.8
ppc64el create device mappings for partitions
ii kpartx-boot 0.4.9-3ubuntu7.8
all Provides kpartx during boot
ii multipath-tools 0.4.9-3ubuntu7.8
ppc64el maintain multipath block device access
ii multipath-tools-boot 0.4.9-3ubuntu7.8
all Support booting from multipath devices
Notice the 2 LPARs (pinelp2 and pinelp3) booted from an individual path.
On pinelp2, it was required to manually include the rootfs device's WWID
(below).
root@pinelp2:~# mount | grep ' / '
/dev/sdq2 on / type ext4 (rw,errors=remount-ro)
root@pinelp3:~# mount | grep ' / '
/dev/sdg2 on / type ext4 (rw,errors=remount-ro)
Make sure the rootfs device is listed in /etc/multipath/wwids in the
initramfs:
- This is required on pinelp2:
root@pinelp2:~# multipath -c /dev/sdq
/dev/sdq is not a valid multipath device path
root@pinelp2:~# multipath -d -v3 | grep 'sdq: uid ='
Feb 05 11:04:14 | sdq: uid = 36005076304ffc4410000000000000161 (callout)
root@pinelp2:~# multipath -d -v3 | grep 'sdq: uid =' | awk '{
print "/" $8 "/" }' >> /etc/multipath/wwids
root@pinelp2:~# multipath -c /dev/sdq
/dev/sdq is a valid multipath device path
root@pinelp2:~# update-initramfs -u
update-initramfs: Generating /boot/initrd.img-4.2.0-27-generic
- Not required on pinelp3:
root@pinelp3:~# multipath -c /dev/sdg
/dev/sdg is a valid multipath device path
Already in WWIDs, so this is likely a timing problem in the boot
(works some of the times).
Test it on both LPARs.
root@pinelp2:~# reboot
<...>
root@pinelp3:~# reboot
<...>
The 2 LPARs booted successfully from a multipath device:
root@pinelp2:~# mount | grep ' / '
/dev/mapper/mpath3-part2 on / type ext4 (rw,errors=remount-ro)
root@pinelp3:~# mount | grep ' / '
/dev/mapper/mpath4-part2 on / type ext4 (rw,errors=remount-ro)
For details: console log for pinelp2:
Notice:
- The scsi disks are added /after/ multipathd is started.
- There *IS* a wait of some seconds at this point.
So, the udevadm settle is indeed working.
Loading, please wait...
[ 0.750797] systemd-udevd[206]: starting version 204
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top
... Begin: Loading multipath modules ... [ 0.772562] device-mapper:
multipath: version 1.9.0 loaded
Success: loaded module dm-multipath.
Failure: failed to load module dm-emc.
done.
Begin: Starting multipathd ... [ 0.778387] multipathd (239):
/proc/239/oom_adj is deprecated, please use /proc/239/oom_score_adj instead.
[ 0.808644] Emulex LightPulse Fibre Channel SCSI driver 10.7.0.0.
[ 0.808672] Copyright(c) 2004-2015 Emulex. All rights reserved.
[ 0.810789] bnx2x: Broadcom NetXtreme II 5771x/578xx 10/20-Gigabit
Ethernet Driver bnx2x 1.710.51-0 (2014/02/10)
[ 0.810928] bnx2x 0000:01:00.0: msix capability found
[ 0.811271] bnx2x 0000:01:00.0: enabling device (0140 -> 0142)
[ 0.815918] lpfc 0001:01:00.0: ibm,query-pe-dma-windows(53) 10000
8000000 20000019 returned 0
[ 0.816230] lpfc 0001:01:00.0: ibm,create-pe-dma-window(54) 10000
8000000 20000019 10 22 returned 0 (liobn = 0x70000019 starting addr = 8000000 0)
[ 0.871171] lpfc 0001:01:00.0: Using 64-bit direct DMA at offset
800000000000000
[ 0.871273] bnx2x 0000:01:00.0: ibm,query-pe-dma-windows(53) 10000
8000000 20000018 returned 0
[ 0.871637] bnx2x 0000:01:00.0: ibm,create-pe-dma-window(54) 10000
8000000 20000018 10 22 returned 0 (liobn = 0x70000018 starting addr = 8000000 0)
[ 0.901703] bnx2x 0000:01:00.0: Using 64-bit direct DMA at offset
800000000000000
[ 0.902119] bnx2x 0000:01:00.0: part number 0-0-0-0
[ 0.984585] bnx2x 0000:01:00.1: msix capability found
[ 0.984794] bnx2x 0000:01:00.1: enabling device (0140 -> 0142)
[ 0.988031] bnx2x 0000:01:00.1: Using 64-bit direct DMA at offset
800000000000000
[ 0.988418] bnx2x 0000:01:00.1: part number 0-0-0-0
[ 1.032097] scsi host0: Emulex LPe16000 16Gb PCIe Fibre Channel
Adapter on PCI bus 01 device 00 irq 506
[ 1.059495] usb 2-4: new SuperSpeed USB device number 2 using
xhci_hcd
[ 1.084352] usb 2-4: New USB device found, idVendor=1a5a,
idProduct=0006
[ 1.084371] usb 2-4: New USB device strings: Mfr=1, Product=2,
SerialNumber=3
[ 1.084375] usb 2-4: Product: RDX-USB3
[ 1.084378] usb 2-4: Manufacturer: IBM T
[ 1.084380] usb 2-4: SerialNumber: 00102000163B
[ 1.087773] usb-storage 2-4:1.0: USB Mass Storage device detected
[ 1.087964] scsi host1: usb-storage 2-4:1.0
[ 1.088040] usbcore: registered new interface driver usb-storage
[ 1.089252] usbcore: registered new interface driver uas
[ 1.596132] lpfc 0001:01:00.1: enabling device (0144 -> 0146)
[ 1.599720] lpfc 0001:01:00.1: Using 64-bit direct DMA at offset
800000000000000
[ 1.760380] scsi host2: Emulex LPe16000 16Gb PCIe Fibre Channel
Adapter on PCI bus 01 device 01 irq 507
[ 2.088013] scsi 1:0:0:0: Direct-Access IBM T RDX-USB3
0195 PQ: 0 ANSI: 3
[ 2.088322] sd 1:0:0:0: Attached scsi generic sg0 type 0
[ 2.088495] sd 1:0:0:0: [sda] 976764976 512-byte logical blocks:
(500 GB/465 GiB)
[ 2.088947] sd 1:0:0:0: [sda] Write Protect is off
[ 2.089453] sd 1:0:0:0: [sda] No Caching mode page found
[ 2.089457] sd 1:0:0:0: [sda] Assuming drive cache: write through
[ 2.313474] sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 sda8 sda9
sda10 sda11 sda12 sda13 sda14 sda15 >
[ 2.315844] sd 1:0:0:0: [sda] Attached SCSI removable disk
[ 2.710413] lpfc 0001:01:00.0: 0:1303 Link Up Event x1 received
Data: x1 x1 x20 x2 x0 x0 0
[ 2.888505] scsi 0:0:0:0: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 2.890233] sd 0:0:0:0: Attached scsi generic sg1 type 0
[ 2.892179] sd 0:0:0:0: [sdb] 62914560 512-byte logical blocks:
(32.2 GB/30.0 GiB)
[ 2.892767] scsi 0:0:0:1: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 2.893855] sd 0:0:0:0: [sdb] Write Protect is off
[ 2.893965] sd 0:0:0:1: Attached scsi generic sg2 type 0
[ 2.894516] sd 0:0:0:0: [sdb] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 2.894672] sd 0:0:0:1: [sdc] 67108864 512-byte logical blocks:
(34.3 GB/32.0 GiB)
[ 2.895084] scsi 0:0:0:2: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 2.896117] sd 0:0:0:1: [sdc] Write Protect is off
[ 2.896245] sd 0:0:0:2: Attached scsi generic sg3 type 0
[ 2.896710] sd 0:0:0:1: [sdc] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 2.896868] sd 0:0:0:2: [sdd] 71303168 512-byte logical blocks:
(36.5 GB/34.0 GiB)
[ 2.898720] sd 0:0:0:2: [sdd] Write Protect is off
[ 2.899630] random: nonblocking pool is initialized
[ 2.899644] sdb: sdb1 sdb2 sdb3 sdb4
[ 2.900083] sd 0:0:0:2: [sdd] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 2.903626] sd 0:0:0:0: [sdb] Attached SCSI disk
[ 2.904561] scsi 0:0:0:3: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 2.911458] sd 0:0:0:3: Attached scsi generic sg4 type 0
[ 2.911913] sdc: sdc1 sdc2 sdc3 sdc4
[ 2.913019] sd 0:0:0:3: [sde] 75497472 512-byte logical blocks:
(38.6 GB/36.0 GiB)
[ 2.915943] sdd: sdd1 sdd2 sdd3 sdd4
[ 2.918011] sd 0:0:0:3: [sde] Write Protect is off
[ 2.920584] sd 0:0:0:3: [sde] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 2.923196] sd 0:0:0:1: [sdc] Attached SCSI disk
[ 2.924875] sd 0:0:0:2: [sdd] Attached SCSI disk
[ 2.925016] scsi 0:0:1:0: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 2.933769] sd 0:0:1:0: Attached scsi generic sg5 type 0
[ 2.941785] sd 0:0:1:0: [sdf] 62914560 512-byte logical blocks:
(32.2 GB/30.0 GiB)
[ 2.941905] sde: sde1 sde2 sde3
[ 2.952749] sd 0:0:0:3: [sde] Attached SCSI disk
[ 2.960043] scsi 0:0:1:1: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 2.968137] sd 0:0:1:0: [sdf] Write Protect is off
[ 2.974386] sd 0:0:1:1: Attached scsi generic sg6 type 0
[ 2.980601] sd 0:0:1:0: [sdf] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 2.980624] sd 0:0:1:1: [sdg] 67108864 512-byte logical blocks:
(34.3 GB/32.0 GiB)
[ 2.985012] scsi 0:0:1:2: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 2.998605] sd 0:0:1:1: [sdg] Write Protect is off
[ 2.998806] sd 0:0:1:2: Attached scsi generic sg7 type 0
[ 3.002843] sd 0:0:1:2: [sdh] 71303168 512-byte logical blocks:
(36.5 GB/34.0 GiB)
[ 3.004300] sd 0:0:1:1: [sdg] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.006821] scsi 0:0:1:3: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 3.029852] sd 0:0:1:2: [sdh] Write Protect is off
[ 3.030067] sd 0:0:1:3: Attached scsi generic sg8 type 0
[ 3.036525] sdf: sdf1 sdf2 sdf3 sdf4
[ 3.039252] sd 0:0:1:3: [sdi] 75497472 512-byte logical blocks:
(38.6 GB/36.0 GiB)
[ 3.039392] sd 0:0:1:2: [sdh] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.043288] scsi 0:0:2:0: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 3.091144] sd 0:0:1:3: [sdi] Write Protect is off
[ 3.091449] sd 0:0:2:0: Attached scsi generic sg9 type 0
[ 3.100773] sdg: sdg1 sdg2 sdg3 sdg4
[ 3.105097] sd 0:0:2:0: [sdj] 62914560 512-byte logical blocks:
(32.2 GB/30.0 GiB)
[ 3.109152] sd 0:0:1:3: [sdi] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.123279] scsi 0:0:2:1: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 3.123299] sd 0:0:1:0: [sdf] Attached SCSI disk
[ 3.135917] sd 0:0:2:0: [sdj] Write Protect is off
[ 3.145103] sd 0:0:2:1: Attached scsi generic sg10 type 0
[ 3.145454] sdh: sdh1 sdh2 sdh3 sdh4
[ 3.158675] sd 0:0:2:0: [sdj] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.158707] sd 0:0:2:1: [sdk] 67108864 512-byte logical blocks:
(34.3 GB/32.0 GiB)
[ 3.169978] scsi 0:0:2:2: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 3.176830] sd 0:0:1:1: [sdg] Attached SCSI disk
[ 3.193476] sd 0:0:2:1: [sdk] Write Protect is off
[ 3.193885] sd 0:0:2:2: Attached scsi generic sg11 type 0
[ 3.202542] sd 0:0:2:2: [sdl] 71303168 512-byte logical blocks:
(36.5 GB/34.0 GiB)
[ 3.202740] sdi: sdi1 sdi2 sdi3
[ 3.209505] sd 0:0:2:1: [sdk] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.209521] scsi 0:0:2:3: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 3.213789] sd 0:0:1:2: [sdh] Attached SCSI disk
[ 3.233428] sd 0:0:2:2: [sdl] Write Protect is off
[ 3.233706] sd 0:0:2:3: Attached scsi generic sg12 type 0
[ 3.237928] sdj: sdj1 sdj2 sdj3 sdj4
[ 3.239901] sd 0:0:2:3: [sdm] 75497472 512-byte logical blocks:
(38.6 GB/36.0 GiB)
[ 3.250975] sd 0:0:2:2: [sdl] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.251285] scsi 0:0:3:0: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 3.272793] sd 0:0:1:3: [sdi] Attached SCSI disk
[ 3.286026] sd 0:0:2:3: [sdm] Write Protect is off
[ 3.286374] sd 0:0:3:0: Attached scsi generic sg13 type 0
[ 3.303858] sd 0:0:3:0: [sdn] 62914560 512-byte logical blocks:
(32.2 GB/30.0 GiB)
[ 3.307006] sdk: sdk1 sdk2 sdk3 sdk4
[ 3.313802] sd 0:0:2:3: [sdm] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.335800] sd 0:0:2:0: [sdj] Attached SCSI disk
[ 3.342298] scsi 0:0:3:1: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 3.384744] sd 0:0:3:0: [sdn] Write Protect is off
[ 3.392239] sdl: sdl1 sdl2 sdl3 sdl4
[ 3.400639] sd 0:0:3:1: Attached scsi generic sg14 type 0
[ 3.418066] sd 0:0:3:0: [sdn] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.418077] sd 0:0:3:1: [sdo] 67108864 512-byte logical blocks:
(34.3 GB/32.0 GiB)
[ 3.426614] sd 0:0:2:1: [sdk] Attached SCSI disk
[ 3.426666] scsi 0:0:3:2: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 3.477763] sd 0:0:3:1: [sdo] Write Protect is off
[ 3.478055] sd 0:0:3:2: Attached scsi generic sg15 type 0
[ 3.491598] sdm: sdm1 sdm2 sdm3
[ 3.508324] sd 0:0:3:2: [sdp] 71303168 512-byte logical blocks:
(36.5 GB/34.0 GiB)
[ 3.523514] sd 0:0:2:2: [sdl] Attached SCSI disk
[ 3.531090] sd 0:0:3:1: [sdo] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.531182] scsi 0:0:3:3: Direct-Access IBM 2107900
.234 PQ: 0 ANSI: 5
[ 3.584206] sd 0:0:3:2: [sdp] Write Protect is off
[ 3.594796] sd 0:0:3:3: Attached scsi generic sg16 type 0
[ 3.604766] sdn: sdn1 sdn2 sdn3 sdn4
[ 3.613399] sd 0:0:3:3: [sdq] 75497472 512-byte logical blocks:
(38.6 GB/36.0 GiB)
[ 3.622867] sd 0:0:3:2: [sdp] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.652150] sd 0:0:2:3: [sdm] Attached SCSI disk
[ 3.713607] sd 0:0:3:3: [sdq] Write Protect is off
[ 3.748953] sdo: sdo1 sdo2 sdo3 sdo4
[ 3.758987] sd 0:0:3:3: [sdq] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3.807023] sd 0:0:3:0: [sdn] Attached SCSI disk
[ 3.854193] sdp: sdp1 sdp2 sdp3 sdp4
[ 3.920408] sd 0:0:3:1: [sdo] Attached SCSI disk
[ 3.971590] sdq: sdq1 sdq2 sdq3
[ 4.010942] sd 0:0:3:2: [sdp] Attached SCSI disk
[ 4.089117] device-mapper: multipath round-robin: version 1.0.0
loaded
[ 4.128373] sd 0:0:3:3: [sdq] Attached SCSI disk
done.
Begin: Running /scripts/local-premount ... [ 5.223400] raid6:
altivecx1 gen() 7409 MB/s
[ 5.291390] raid6: altivecx2 gen() 14423 MB/s
[ 5.359390] raid6: altivecx4 gen() 20551 MB/s
[ 5.427397] raid6: altivecx8 gen() 11135 MB/s
[ 5.495395] raid6: int64x1 gen() 3714 MB/s
[ 5.563411] raid6: int64x1 xor() 1300 MB/s
[ 5.631390] raid6: int64x2 gen() 7399 MB/s
[ 5.699401] raid6: int64x2 xor() 2360 MB/s
[ 5.767392] raid6: int64x4 gen() 9980 MB/s
[ 5.835389] raid6: int64x4 xor() 2058 MB/s
[ 5.903408] raid6: int64x8 gen() 3289 MB/s
[ 5.971401] raid6: int64x8 xor() 1296 MB/s
[ 5.971408] raid6: using algorithm altivecx4 gen() 20551 MB/s
[ 5.971410] raid6: using intx1 recovery algorithm
[ 5.972320] xor: measuring software checksum speed
[ 6.011392] 8regs : 15696.000 MB/sec
[ 6.051392] 8regs_prefetch: 14160.000 MB/sec
[ 6.091394] 32regs : 16288.000 MB/sec
[ 6.131392] 32regs_prefetch: 15008.000 MB/sec
[ 6.171394] altivec : 22656.000 MB/sec
[ 6.171401] xor: using function: altivec (22656.000 MB/sec)
[ 6.196640] Btrfs loaded
Scanning for Btrfs filesystems
Begin: Waiting for udev to settle (multipath) ... <<<... WAITING ...>>>
done.
done.
[ 20.233694] EXT4-fs (dm-17): mounted filesystem with ordered data
mode. Opts: (null)
Begin: Running /scripts/local-bottom ... Begin: Stopping multipathd ...
done.
done.
done.
Begin: Running /scripts/init-bottom ...
done.
[ 21.427248] init: plymouth-upstart-bridge main process (1881)
terminated with status 1
Ubuntu 14.04.3 LTS pinelp2 hvc0
pinelp2 login: root
Password:
Last login: Fri Feb 5 10:43:08 CST 2016 from
lte.isst.aus.stglabs.ibm.com on pts/8
Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 4.2.0-27-generic ppc64le)
<...>
root@pinelp2:~# mount | grep ' / '
/dev/mapper/mpath3-part2 on / type ext4 (rw,errors=remount-ro)
** Tags removed: verification-needed
** Tags added: verification-done
--
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to multipath-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1526984
Title:
ISST-LTE: root mpath device unavailable after installation
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1526984/+subscriptions
--
Ubuntu-server-bugs mailing list
[email protected]
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
