[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Launchpad Bug Tracker]

This bug was fixed in the package linux - 4.4.0-93.116

---
linux (4.4.0-93.116) xenial; urgency=low

  * linux: 4.4.0-93.116 -proposed tracker (LP: #1709296)

  * Creating conntrack entry failure with kernel 4.4.0-89 (LP: #1709032)
- Revert "Revert "netfilter: synproxy: fix conntrackd interaction""
- netfilter: nf_ct_ext: fix possible panic after nf_ct_extend_unregister

  * CVE-2017-1000112
- Revert "udp: consistently apply ufo or fragmentation"
- udp: consistently apply ufo or fragmentation

  * CVE-2017-1000111
- Revert "net-packet: fix race in packet_set_ring on PACKET_RESERVE"
- packet: fix tp_reserve race in packet_set_ring

  * kernel BUG at [tty_ldisc_reinit] mm/slub.c! (LP: #1709126)
- tty: Simplify tty_set_ldisc() exit handling
- tty: Reset c_line from driver's init_termios
- tty: Handle NULL tty->ldisc
- tty: Move tty_ldisc_kill()
- tty: Use 'disc' for line discipline index name
- tty: Refactor tty_ldisc_reinit() for reuse
- tty: Destroy ldisc instance on hangup

  * atheros bt failed after S3 (LP: #1706833)
- SAUCE: Bluetooth: Make request workqueue freezable

  * The Precision Touchpad(PTP) button sends incorrect event code (LP: #1708372)
- HID: multitouch: handle external buttons for Precision Touchpads

  * Set CONFIG_SATA_HIGHBANK=y on armhf (LP: #1703430)
- [Config] CONFIG_SATA_HIGHBANK=y

  * xfs slab objects (memory) leak when xfs shutdown is called (LP: #1706132)
- xfs: fix xfs_log_ticket leak in xfs_end_io() after fs shutdown

  * Adt tests of src:linux time out often on armhf lxc containers (LP: #1705495)
- [Packaging] tests -- reduce rebuild test to one flavour

  * CVE-2017-7495
- ext4: fix data exposure after a crash

  * ubuntu/rsi driver downlink wifi throughput drops to 5-6 Mbps when BT
keyboard is connected (LP: #1706991)
- SAUCE: Redpine: enable power save by default for coex mode
- SAUCE: Redpine: uapsd configuration changes

  * [Hyper-V] hv_netvsc: Exclude non-TCP port numbers from vRSS hashing
(LP: #1690174)
- hv_netvsc: Exclude non-TCP port numbers from vRSS hashing

  * ath10k doesn't report full RSSI information (LP: #1706531)
- ath10k: add per chain RSSI reporting

  * ideapad_laptop don't support v310-14isk (LP: #1705378)
- platform/x86: ideapad-laptop: Add several models to no_hw_rfkill

  * [8087:0a2b] Failed to load bluetooth firmware(might affect some other Intel
bt devices) (LP: #1705633)
- Bluetooth: btintel: Create common Intel Version Read function
- Bluetooth: Use switch statement for Intel hardware variants
- Bluetooth: Replace constant hw_variant from Intel Bluetooth firmware
  filename
- Bluetooth: hci_intel: Fix firmware file name to use hw_variant
- Bluetooth: btintel: Add MODULE_FIRMWARE entries for iBT 3.5 controllers

  * xhci_hcd: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2
comp_code 13 (LP: #1667750)
- xhci: Bad Ethernet performance plugged in ASM1042A host

  * OpenPower: Some multipaths temporarily have only a single path
(LP: #1696445)
- scsi: ses: don't get power status of SES device slot on probe

  * Hotkeys on new Thinkpad systems aren't working (LP: #1705169)
- platform/x86: thinkpad_acpi: Adding new hotkey ID for Lenovo thinkpad
- platform/x86: thinkpad_acpi: guard generic hotkey case
- platform/x86: thinkpad_acpi: add mapping for new hotkeys

  * CVE-2015-7837
- SAUCE: (no-up) kexec/uefi: copy secure_boot flag in boot params across 
kexec
  reboot

  * misleading kernel warning skb_warn_bad_offload during checksum calculation
(LP: #1705447)
- net: reduce skb_warn_bad_offload() noise

  * bonding: stack dump when unregistering a netdev (LP: #1704102)
- bonding: avoid NETDEV_CHANGEMTU event when unregistering slave

  * Ubuntu 16.04 IOB Error when the Mustang board rebooted (LP: #1693673)
- drivers: net: xgene: Fix redundant prefetch buffer cleanup

  * Ubuntu16.04: NVMe 4K+T10 DIF/DIX format returns I/O error on dd with split
op (LP: #1689946)
- blk-mq: NVMe 512B/4K+T10 DIF/DIX format returns I/O error on dd with split
  op

  * linux >= 4.2: bonding 802.3ad does not work with 5G, 25G and 50G link speeds
(LP: #1697892)
- bonding: add 802.3ad support for 100G speeds
- bonding: fix 802.3ad aggregator reselection
- bonding: add 802.3ad support for 25G speeds
- bonding: fix 802.3ad support for 5G and 50G speeds

  * Xenial update to 4.4.79 stable release (LP: #1707233)
- disable new gcc-7.1.1 warnings for now
- ir-core: fix gcc-7 warning on bool arithmetic
- s5p-jpeg: don't return a random width/height
- thermal: cpu_cooling: Avoid accessing potentially freed structures
- ath9k: fix tx99 use after free
- ath9k: fix tx99 bus error
- NFC: fix broken device allocation
- NFC: nfcmrvl_uart: add missing tty-device sanity check
- NFC: nfcmrvl: do not use 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Kleber Sacilotto de Souza]

Hi Mauricio,

Could you please verify the fix with the xenial kernel currently in
-proposed?

Thank you!

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Xenial:
  Fix Committed
Status in linux source package in Yakkety:
  Fix Released
Status in linux source package in Zesty:
  Fix Released

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  252
  root@smb1p1:~#

  ...

  - After 47 minutes, multipath -ll command still shows some paths with

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Kleber Sacilotto de Souza]

This bug is awaiting verification that the kernel in -proposed solves
the problem. Please test the kernel and update this bug with the
results. If the problem is solved, change the tag 'verification-needed-
xenial' to 'verification-done-xenial'. If the problem still exists,
change the tag 'verification-needed-xenial' to 'verification-failed-
xenial'.

If verification is not done by 5 working days from today, this fix will
be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how
to enable and use -proposed. Thank you!


** Tags added: verification-needed-xenial

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Xenial:
  Fix Committed
Status in linux source package in Yakkety:
  Fix Released
Status in linux source package in Zesty:
  Fix Released

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Kleber Sacilotto de Souza]

** Changed in: linux (Ubuntu Xenial)
   Status: Triaged => Fix Committed

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Xenial:
  Fix Committed
Status in linux source package in Yakkety:
  Fix Released
Status in linux source package in Zesty:
  Fix Released

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  252
  root@smb1p1:~#

  ...

  - After 47 minutes, multipath -ll command still shows some paths with
  only single path and no 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Joseph Salisbury]

** Changed in: linux (Ubuntu Xenial)
   Importance: Undecided => Medium

** Changed in: linux (Ubuntu Xenial)
   Status: New => Triaged

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Xenial:
  Triaged
Status in linux source package in Yakkety:
  Fix Released
Status in linux source package in Zesty:
  Fix Released

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  252
  root@smb1p1:~#

  ...

  - After 47 minutes, multipath -ll 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Stefan Bader]

** Also affects: linux (Ubuntu Xenial)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Xenial:
  New
Status in linux source package in Yakkety:
  Fix Released
Status in linux source package in Zesty:
  Fix Released

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  252
  root@smb1p1:~#

  ...

  - After 47 minutes, multipath -ll command still shows some paths with
  only single path and no 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Frank Heimes]

** Changed in: ubuntu-power-systems
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Yakkety:
  Fix Released
Status in linux source package in Zesty:
  Fix Released

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  252
  root@smb1p1:~#

  ...

  - After 47 minutes, multipath -ll command still shows some paths with
  only single path and no redundancy.

  
  root@smb1p1:~# multipath -ll | grep 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Launchpad Bug Tracker]

This bug was fixed in the package linux - 4.10.0-28.32

---
linux (4.10.0-28.32) zesty; urgency=low

  * linux: 4.10.0-28.32 -proposed tracker (LP: #1701013)

  * KILLER1435-S[0489:e0a2] BT cannot search BT 4.0 device (LP: #1699651)
- Bluetooth: btusb: Add support for 0489:e0a2 QCA_ROME device

  * aacraid driver may return uninitialized stack data to userspace
(LP: #1700077)
- SAUCE: scsi: aacraid: Don't copy uninitialized stack memory to userspace

  * CVE-2017-9605
- drm/vmwgfx: Make sure backup_handle is always valid

  * CVE-2017-1000380
- ALSA: timer: Fix race between read and ioctl
- ALSA: timer: Fix missing queue indices reset at SNDRV_TIMER_IOCTL_SELECT

  * XDP eBPF programs fail to verify on Zesty ppc64el (LP: #1699627)
- [Config] ppc64el: build for Power8 not Power7

  * AACRAID for power9 platform (LP: #1689980)
- scripts/spelling.txt: add "therfore" pattern and fix typo instances
- scsi: aacraid: fix PCI error recovery path
- scsi: aacraid: pci_alloc_consistent() failures on ARM64
- scsi: aacraid: Remove __GFP_DMA for raw srb memory
- scsi: aacraid: Fix DMAR issues with iommu=pt
- scsi: aacraid: Added 32 and 64 queue depth for arc natives
- scsi: aacraid: Set correct Queue Depth for HBA1000 RAW disks
- scsi: aacraid: Remove reset support from check_health
- scsi: aacraid: Change wait time for fib completion
- scsi: aacraid: Log count info of scsi cmds before reset
- scsi: aacraid: Print ctrl status before eh reset
- scsi: aacraid: Using single reset mask for IOP reset
- scsi: aacraid: Rework IOP reset
- scsi: aacraid: Add periodic checks to see IOP reset status
- scsi: aacraid: Rework SOFT reset code
- scsi: aacraid: Rework aac_src_restart
- scsi: aacraid: Use correct function to get ctrl health
- scsi: aacraid: Make sure ioctl returns on controller reset
- scsi: aacraid: Enable ctrl reset for both hba and arc
- scsi: aacraid: Add reset debugging statements
- scsi: aacraid: Remove reference to Series-9
- scsi: aacraid: Update driver version to 50834

  * arm64 kernel crashdump support (LP: #1694859)
- memblock: add memblock_clear_nomap()
- memblock: add memblock_cap_memory_range()
- arm64: limit memory regions based on DT property, usable-memory-range
- arm64: kdump: reserve memory for crash dump kernel
- arm64: mm: add set_memory_valid()
- arm64: mm: use phys_addr_t instead of unsigned long in __map_memblock
- arm64: kdump: protect crash dump kernel memory
- arm64: hibernate: preserve kdump image around hibernation
- arm64: kdump: implement machine_crash_shutdown()
- arm64: kdump: add VMCOREINFO's for user-space tools
- [Config] CONFIG_CRASH_DUMP=y on arm64
- arm64: kdump: provide /proc/vmcore file
- Documentation: kdump: describe arm64 port
- Documentation: dt: chosen properties for arm64 kdump
- efi/libstub/arm*: Set default address and size cells values for an empty 
dtb

  * hibmc driver does not include "pci:" prefix in bus ID (LP: #1698700)
- SAUCE: drm: hibmc: Use set_busid function from drm core

  * Processes in "D" state due to zap_pid_ns_processes kernel call with Ubuntu +
Docker (LP: #1698264)
- pid_ns: Sleep in TASK_INTERRUPTIBLE in zap_pid_ns_processes

  * Bugfixes for  hns network driver (LP: #1696031)
- hns_enet: use cpumask_var_t for on-stack mask
- net: hns: fix uninitialized data use
- net: hns: avoid gcc-7.0.1 warning for uninitialized data
- net: hns: Add ACPI support to check SFP present
- net: hns: Fix the implementation of irq affinity function
- net: hns: Modify GMAC init TX threshold value
- net: hns: Optimize the code for GMAC pad and crc Config
- net: hns: Remove redundant memset during buffer release
- net: hns: bug fix of ethtool show the speed
- net: hns: Optimize hns_nic_common_poll for better performance
- net: hns: Fix to adjust buf_size of ring according to mtu
- net: hns: Replace netif_tx_lock to ring spin lock
- net: hns: Correct HNS RSS key set function
- net: hns: Remove the redundant adding and deleting mac function
- net: hns: Remove redundant mac_get_id()
- net: hns: Remove redundant mac table operations
- net: hns: Clean redundant code from hns_mdio.c file
- net: hns: Optimise the code in hns_mdio_wait_ready()
- net: hns: Simplify the exception sequence in hns_ppe_init()
- net: hns: Adjust the SBM module buffer threshold
- net: hns: Avoid Hip06 chip TX packet line bug
- net: hns: Some checkpatch.pl script & warning fixes
- net: hns: support deferred probe when can not obtain irq
- net: hns: support deferred probe when no mdio
- net: hns: fix ethtool_get_strings overflow in hns driver

  * CVE-2017-7346
- drm/vmwgfx: limit the number of mip levels in 
vmw_gb_surface_define_ioctl()

  * [SRU][Zesty] qcom_emac is unable to get ip address with at803x 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Launchpad Bug Tracker]

This bug was fixed in the package linux - 4.8.0-59.64

---
linux (4.8.0-59.64) yakkety; urgency=low

  * linux: 4.8.0-59.64 -proposed tracker (LP: #1701019)

  * KILLER1435-S[0489:e0a2] BT cannot search BT 4.0 device (LP: #1699651)
- Bluetooth: btusb: Add support for 0489:e0a2 QCA_ROME device

  * CVE-2017-7895
- nfsd4: minor NFSv2/v3 write decoding cleanup
- nfsd: stricter decoding of write-like NFSv2/v3 ops

  * CVE-2017-5551
- tmpfs: clear S_ISGID when setting posix ACLs

  * CVE-2017-9605
- drm/vmwgfx: Make sure backup_handle is always valid

  * CVE-2017-1000380
- ALSA: timer: Fix race between read and ioctl
- ALSA: timer: Fix missing queue indices reset at SNDRV_TIMER_IOCTL_SELECT

  * CVE-2017-9150
- bpf: don't let ldimm64 leak map addresses on unprivileged

  * CVE-2017-5576
- drm/vc4: Fix an integer overflow in temporary allocation layout.

  * Processes in "D" state due to zap_pid_ns_processes kernel call with Ubuntu +
Docker (LP: #1698264)
- pid_ns: Sleep in TASK_INTERRUPTIBLE in zap_pid_ns_processes

  * CVE-2016-9755
- netfilter: ipv6: nf_defrag: drop mangled skb on ream error

  * CVE-2017-7346
- drm/vmwgfx: limit the number of mip levels in 
vmw_gb_surface_define_ioctl()

  * CVE-2017-8924
- USB: serial: io_ti: fix information leak in completion handler

  * CVE-2017-8925
- USB: serial: omninet: fix reference leaks at open

  * CVE-2017-9074
- ipv6: Check ip6_find_1stfragopt() return value properly.

  * CVE-2014-9900
- net: Zeroing the structure ethtool_wolinfo in ethtool_get_wol()

  * OpenPower: Some multipaths temporarily have only a single path
(LP: #1696445)
- scsi: ses: don't get power status of SES device slot on probe

 -- Thadeu Lima de Souza Cascardo   Thu, 29 Jun
2017 14:34:32 -0300

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Committed
Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Yakkety:
  Fix Released
Status in linux source package in Zesty:
  Fix Committed

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Launchpad Bug Tracker]

This bug was fixed in the package linux - 4.8.0-59.64

---
linux (4.8.0-59.64) yakkety; urgency=low

  * linux: 4.8.0-59.64 -proposed tracker (LP: #1701019)

  * KILLER1435-S[0489:e0a2] BT cannot search BT 4.0 device (LP: #1699651)
- Bluetooth: btusb: Add support for 0489:e0a2 QCA_ROME device

  * CVE-2017-7895
- nfsd4: minor NFSv2/v3 write decoding cleanup
- nfsd: stricter decoding of write-like NFSv2/v3 ops

  * CVE-2017-5551
- tmpfs: clear S_ISGID when setting posix ACLs

  * CVE-2017-9605
- drm/vmwgfx: Make sure backup_handle is always valid

  * CVE-2017-1000380
- ALSA: timer: Fix race between read and ioctl
- ALSA: timer: Fix missing queue indices reset at SNDRV_TIMER_IOCTL_SELECT

  * CVE-2017-9150
- bpf: don't let ldimm64 leak map addresses on unprivileged

  * CVE-2017-5576
- drm/vc4: Fix an integer overflow in temporary allocation layout.

  * Processes in "D" state due to zap_pid_ns_processes kernel call with Ubuntu +
Docker (LP: #1698264)
- pid_ns: Sleep in TASK_INTERRUPTIBLE in zap_pid_ns_processes

  * CVE-2016-9755
- netfilter: ipv6: nf_defrag: drop mangled skb on ream error

  * CVE-2017-7346
- drm/vmwgfx: limit the number of mip levels in 
vmw_gb_surface_define_ioctl()

  * CVE-2017-8924
- USB: serial: io_ti: fix information leak in completion handler

  * CVE-2017-8925
- USB: serial: omninet: fix reference leaks at open

  * CVE-2017-9074
- ipv6: Check ip6_find_1stfragopt() return value properly.

  * CVE-2014-9900
- net: Zeroing the structure ethtool_wolinfo in ethtool_get_wol()

  * OpenPower: Some multipaths temporarily have only a single path
(LP: #1696445)
- scsi: ses: don't get power status of SES device slot on probe

 -- Thadeu Lima de Souza Cascardo   Thu, 29 Jun
2017 14:34:32 -0300

** Changed in: linux (Ubuntu Yakkety)
   Status: Fix Committed => Fix Released

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2014-9900

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2016-9755

** CVE added: https://cve.mitre.org/cgi-
bin/cvename.cgi?name=2017-1000380

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2017-5551

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2017-5576

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2017-7346

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2017-7895

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2017-8924

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2017-8925

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2017-9074

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2017-9150

** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2017-9605

** Changed in: linux (Ubuntu Yakkety)
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Committed
Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Yakkety:
  Fix Released
Status in linux source package in Zesty:
  Fix Committed

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Launchpad Bug Tracker]

This bug was fixed in the package linux - 4.11.0-10.15

---
linux (4.11.0-10.15) artful; urgency=low

  * linux: 4.11.0-10.15 -proposed tracker (LP: #1701271)

  * Artful update to v4.11.8 stable release (LP: #1701269)
- clk: sunxi-ng: a31: Correct lcd1-ch1 clock register offset
- clk: sunxi-ng: v3s: Fix usb otg device reset bit
- clk: sunxi-ng: sun5i: Fix ahb_bist_clk definition
- xen/blkback: fix disconnect while I/Os in flight
- xen-blkback: don't leak stack data via response ring
- ALSA: firewire-lib: Fix stall of process context at packet error
- ALSA: pcm: Don't treat NULL chmap as a fatal error
- ALSA: hda - Add Coffelake PCI ID
- ALSA: hda - Apply quirks to Broxton-T, too
- fs/exec.c: account for argv/envp pointers
- powerpc/perf: Fix oops when kthread execs user process
- autofs: sanity check status reported with AUTOFS_DEV_IOCTL_FAIL
- fs/dax.c: fix inefficiency in dax_writeback_mapping_range()
- lib/cmdline.c: fix get_options() overflow while parsing ranges
- perf/x86/intel: Add 1G DTLB load/store miss support for SKL
- perf probe: Fix probe definition for inlined functions
- KVM: x86: fix singlestepping over syscall
- KVM: MIPS: Fix maybe-uninitialized build failure
- KVM: s390: gaccess: fix real-space designation asce handling for gmap
  shadows
- KVM: PPC: Book3S HV: Cope with host using large decrementer mode
- KVM: PPC: Book3S HV: Preserve userspace HTM state properly
- KVM: PPC: Book3S HV: Ignore timebase offset on POWER9 DD1
- KVM: PPC: Book3S HV: Context-switch EBB registers properly
- KVM: PPC: Book3S HV: Restore critical SPRs to host values on guest exit
- KVM: PPC: Book3S HV: Save/restore host values of debug registers
- CIFS: Improve readdir verbosity
- CIFS: Fix some return values in case of error in 'crypt_message'
- cxgb4: notify uP to route ctrlq compl to rdma rspq
- HID: Add quirk for Dell PIXART OEM mouse
- random: silence compiler warnings and fix race
- signal: Only reschedule timers on signals timers have sent
- powerpc/kprobes: Pause function_graph tracing during jprobes handling
- powerpc/64s: Handle data breakpoints in Radix mode
- Input: i8042 - add Fujitsu Lifebook AH544 to notimeout list
- brcmfmac: add parameter to pass error code in firmware callback
- brcmfmac: use firmware callback upon failure to load
- brcmfmac: unbind all devices upon failure in firmware callback
- time: Fix clock->read(clock) race around clocksource changes
- time: Fix CLOCK_MONOTONIC_RAW sub-nanosecond accounting
- arm64/vdso: Fix nsec handling for CLOCK_MONOTONIC_RAW
- target: Fix kref->refcount underflow in transport_cmd_finish_abort
- iscsi-target: Fix delayed logout processing greater than
  SECONDS_FOR_LOGOUT_COMP
- iscsi-target: Reject immediate data underflow larger than SCSI transfer
  length
- drm/radeon: add a PX quirk for another K53TK variant
- drm/radeon: add a quirk for Toshiba Satellite L20-183
- drm/amdgpu/atom: fix ps allocation size for EnableDispPowerGating
- drm/amdgpu: adjust default display clock
- drm/amdgpu: add Polaris12 DID
- ACPI / scan: Apply default enumeration to devices with ACPI drivers
- ACPI / scan: Fix enumeration for special SPI and I2C devices
- rxrpc: Fix several cases where a padded len isn't checked in ticket decode
- drm: Fix GETCONNECTOR regression
- usb: gadget: f_fs: avoid out of bounds access on comp_desc
- spi: double time out tolerance
- net: phy: fix marvell phy status reading
- brcmfmac: fix uninitialized warning in brcmf_usb_probe_phase2()
- Linux 4.11.8

  * powerpc: Invalidate ERAT on powersave wakeup for POWER9 (LP: #1700521)
- SAUCE: powerpc: Invalidate ERAT on powersave wakeup for POWER9

  * Miscellaneous Ubuntu changes
- d-i: Move qcom-emac from arm64 to shared nic-modules

 -- Seth Forshee   Thu, 29 Jun 2017 08:46:53
-0500

** Changed in: linux (Ubuntu)
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Committed
Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Yakkety:
  Fix Committed
Status in linux source package in Zesty:
  Fix Committed

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[bugproxy]

--- Comment From mauri...@br.ibm.com 2017-07-11 12:07 EDT---
Marking verification done for zesty and yakkety.

Unfortunately the hardware to verify this is not available anymore, but
the patch is trivial and quite contained, so the code inspection of the
diffs in -proposed revealed it is present and correct.

https://launchpadlibrarian.net/326107857/linux_4.8.0-58.63_4.8.0-59.64.diff.gz
https://launchpadlibrarian.net/326184317/linux_4.10.0-26.30_4.10.0-28.32.diff.gz

Both contain the 3 chunks from the patch in

https://lists.ubuntu.com/archives/kernel-team/2017-June/084718.html

cheers,
Mauricio

** Tags removed: verification-needed-yakkety verification-needed-zesty
** Tags added: verification-done-yakkety verification-done-zesty

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Committed
Status in linux package in Ubuntu:
  New
Status in linux source package in Yakkety:
  Fix Committed
Status in linux source package in Zesty:
  Fix Committed

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Kleber Sacilotto de Souza]

This bug is awaiting verification that the kernel in -proposed solves
the problem. Please test the kernel and update this bug with the
results. If the problem is solved, change the tag 'verification-needed-
zesty' to 'verification-done-zesty'. If the problem still exists, change
the tag 'verification-needed-zesty' to 'verification-failed-zesty'.

If verification is not done by 5 working days from today, this fix will
be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how
to enable and use -proposed. Thank you!

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Committed
Status in linux package in Ubuntu:
  New
Status in linux source package in Yakkety:
  Fix Committed
Status in linux source package in Zesty:
  Fix Committed

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Kleber Sacilotto de Souza]

This bug is awaiting verification that the kernel in -proposed solves
the problem. Please test the kernel and update this bug with the
results. If the problem is solved, change the tag 'verification-needed-
yakkety' to 'verification-done-yakkety'. If the problem still exists,
change the tag 'verification-needed-yakkety' to 'verification-failed-
yakkety'.

If verification is not done by 5 working days from today, this fix will
be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how
to enable and use -proposed. Thank you!


** Tags added: verification-needed-yakkety

** Tags added: verification-needed-zesty

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Committed
Status in linux package in Ubuntu:
  New
Status in linux source package in Yakkety:
  Fix Committed
Status in linux source package in Zesty:
  Fix Committed

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Frank Heimes]

** Changed in: ubuntu-power-systems
   Status: New => Fix Committed

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  Fix Committed
Status in linux package in Ubuntu:
  New
Status in linux source package in Yakkety:
  Fix Committed
Status in linux source package in Zesty:
  Fix Committed

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  252
  root@smb1p1:~#

  ...

  - After 47 minutes, multipath -ll command still shows some paths with
  only single path and no redundancy.

  
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Stefan Bader]

** Changed in: linux (Ubuntu Zesty)
   Status: New => Fix Committed

** Changed in: linux (Ubuntu Yakkety)
   Status: New => Fix Committed

** Changed in: linux (Ubuntu Zesty)
   Importance: Undecided => Medium

** Changed in: linux (Ubuntu Yakkety)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  New
Status in linux package in Ubuntu:
  New
Status in linux source package in Yakkety:
  Fix Committed
Status in linux source package in Zesty:
  Fix Committed

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Stefan Bader]

** Also affects: linux (Ubuntu Yakkety)
   Importance: Undecided
   Status: New

** Also affects: linux (Ubuntu Zesty)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  New
Status in linux package in Ubuntu:
  New
Status in linux source package in Yakkety:
  New
Status in linux source package in Zesty:
  New

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  252
  root@smb1p1:~#

  ...

  - After 47 minutes, multipath -ll command still shows some paths with
  only single path and no 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Frank Heimes]

** Also affects: ubuntu-power-systems
   Importance: Undecided
   Status: New

** Changed in: ubuntu-power-systems
 Assignee: (unassigned) => Canonical Kernel Team (canonical-kernel-team)

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in The Ubuntu-power-systems project:
  New
Status in linux package in Ubuntu:
  New

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  252
  root@smb1p1:~#

  ...

  - After 47 minutes, multipath -ll command still shows some paths with
  only single path and no redundancy.

  
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Joseph Salisbury]

** Tags added: kernel-da-key

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1696445

Title:
  OpenPower: Some multipaths temporarily have only a single path

Status in linux package in Ubuntu:
  New

Bug description:
  [Impact]

   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.

   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.

   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.

  [Test Case]

   * Load the module to access the enclosure and its disks; e.g.,

     $ sudo modprobe mpt3sas

   * Notice the interval between the discovery of each disk; e.g., dmesg

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk

   * The interval should be in the same second or so range with the fix.

     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk

  [Regression Potential]

   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.

   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).

  [Other Info]

   * None at this time.

  
  Problem Description:
  

  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.

  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)

  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.

  
  root@smb1p1:~# multipath -ll|grep dm |wc -l
  103

  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
  [Thu Jun  1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  [Thu Jun  1 14:18:40 2017] sd 17:0:102:0: [sdct] Attached SCSI disk
  [Thu Jun  1 14:18:44 2017] sd 17:0:103:0: [sdcu] Attached SCSI disk
  [Thu Jun  1 14:18:54 2017] sd 17:0:105:0: [sdcv] Attached SCSI disk
  [Thu Jun  1 14:18:59 2017] sd 17:0:106:0: [sdcw] Attached SCSI disk
  [Thu Jun  1 14:19:04 2017] sd 17:0:107:0: [sdcx] Attached SCSI disk
  [Thu Jun  1 14:19:09 2017] sd 17:0:108:0: [sdcy] Attached SCSI disk
  [Thu Jun  1 14:19:14 2017] sd 17:0:109:0: [sdcz] Attached SCSI disk
  [Thu Jun  1 14:19:19 2017] sd 17:0:110:0: [sdda] Attached SCSI disk
  root@smb1p1:~#

  ...

  root@smb1p1:~# multipath -ll|grep dm |wc -l
  142
  root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | tail
  [Thu Jun  1 14:21:54 2017] sd 17:0:141:0: [sdee] Attached SCSI disk
  [Thu Jun  1 14:21:58 2017] sd 17:0:142:0: [sdef] Attached SCSI disk
  [Thu Jun  1 14:22:04 2017] sd 17:0:143:0: [sdeg] Attached SCSI disk
  [Thu Jun  1 14:22:08 2017] sd 17:0:144:0: [sdeh] Attached SCSI disk
  [Thu Jun  1 14:22:14 2017] sd 17:0:145:0: [sdei] Attached SCSI disk
  [Thu Jun  1 14:22:18 2017] sd 17:0:146:0: [sdej] Attached SCSI disk
  [Thu Jun  1 14:22:24 2017] sd 17:0:147:0: [sdek] Attached SCSI disk
  [Thu Jun  1 14:22:29 2017] sd 17:0:148:0: [sdel] Attached SCSI disk
  [Thu Jun  1 14:22:34 2017] sd 17:0:149:0: [sdem] Attached SCSI disk
  [Thu Jun  1 14:22:39 2017] sd 17:0:150:0: [sden] Attached SCSI disk
  root@smb1p1:~#

  
  ...

  - After 43  minutes, multipath -ll command shows some paths with only
  single path and no redundancy and some path with multiple paths and
  redundancy.

  root@smb1p1:~# date
  Thu Jun  1 14:43:00 CDT 2017
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  252
  root@smb1p1:~#

  ...

  - After 47 minutes, multipath -ll command still shows some paths with
  only single path and no redundancy.

  
  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  288
  root@smb1p1:~#


  - After 51 minutes after system reboot, looks like all disk are
  discovered and the Multipath is correctly built.

  root@smb1p1:~# multipath -ll | grep -c 'sd[a-z]\+'
  336

  
  == Comment: 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Mauricio Faria de Oliveira]

patch posted to the kernel-team mailing list:

https://lists.ubuntu.com/archives/kernel-team/2017-June/084718.html

** Description changed:

  [Impact]
  
   * The SES driver causes a long delay in disk discovery when
     a large number of disks is present in the disk enclosure,
     which increases with the number of disks attached.
  
   * This delays the addition and visibility of the disk devices
     to userspace, which among other things causes multipath not
     to have multiple paths, actually, until the disk discovery
     eventually/finally finishes.
  
   * The fix significantly shortens the time taken by the SES
     driver to handle disk discovery, causing no extra delays,
     by removing a superfluous SCSI command sent to enclosure.
  
  [Test Case]
  
   * Load the module to access the enclosure and its disks; e.g.,
  
     $ sudo modprobe mpt3sas
  
   * Notice the interval between the discovery of each disk; e.g., dmesg
  
     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
     [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
  
   * The interval should be in the same second or so range with the fix.
  
     $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
     [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
     [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk
  
  [Regression Potential]
  
   * The power status of the disks in the enclosure is no longer
     checked during probe time.  However, the patch demonstrates that
     initial value was never used in any way.  So, little regression
     potential.
  
   * Nonetheless, users of SES enclosures which verify the power status
     of disks in the enclosure might _theoretically_ see a problem, iff
     the fix has a problem (which has not been found yet).
  
  [Other Info]
  
   * None at this time.
  
-  State: Open by: nguyenp on 31 May 2017 15:46:14 
- 
-  Product Name  : OpenPOWER Firmware
-  Product Version   : open-power-SMC-P8DTU-V2.00.GA2-20170126-prod
-  Product Extra :  op-build-3782262
-  Product Extra :  hostboot-7fdfb37
-  Product Extra :  occ-e6e194f
-  Product Extra :  skiboot-5.4.2
-  Product Extra :  linux-4.4.24-openpower1-9641b3a
-  Product Extra :  petitboot-v1.4.0-2f8598b
-  Product Extra :  p8dtu-xml-9a8fee2
- 
- Cable configuration:
- 
- On this P8-Briggs system, I have 2 Seagate Storages running with max 
configuration. There are 84 HDDs drives in each storage. So the total drives is 
168 HDDs for both Seagate storages.
- 
- I connected 2 LSI 9300-8e SAS adapters to 2 Seagate storages with
- alternate cabling for redundancy. See a Figure on the connection below:
- 
- Note:  Each Seagate storage has 2 I/O moudules connection in the back.
-    Both I/O modules from each Seagate does see the same set of HDDs
- 
- Cable connection:
- 
- SAS adapter #1:port1  ->  Seagate #1-A I/O module
-    port0  --> Seagate 
#2-B I/O module
- 
- SAS adapter #2:port1  >  Seagate #2-A I/O module
-    port0  --> Seagate 
#1-B I/O module
- 
- Ubuntu 16.04.2:
- ===
- 
- - Running with new kernel Ubuntu 4.8.0-520-generic #550~16.04.1+bz154734
- from Mauricio Faria De Oliveira.
  
  Problem Description:
  
- In this Briggs system, I'm running with new Ubuntu 4.8.0-520-generic 
#550~16.04.1+bz154734 that has fix for Multipath problem. Mauricio helped to 
patch the system with this kernel last week to fix the multipath io_setup 
failed problem in LTCBug154734.
  
  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.
- 
- == Comment: #13 - Paul Nguyen - 2017-06-01 15:19:58 ==
- 
- -  I agreed with Mauricio that this problem is a timing problem.
- 
- - I re-ran the test and noticed that it took more than 50 minutes after
- system reboot to discover all disks and to build Multipaths correctly.
- 
- - So for it to take this long, it's going to be a problem.
- 
- - I have gathered all logs and attaching to the bug for Mauricio to look
- and confirm.
- 
- - If there is a workaround or fix for faster probe time then I will try
- it out.
- 
- - Below is more information I captured:
  
  Checkpoint #1:
  ==
  - system reboot around 2pm (14:00)
  
  Checkpoint # 2:
  ===
  - It took several minutes for first disk to be detected.
  
- root@smb1p1:~# dmesg -T | grep 'sd 1[78]:' |  grep 'Attached SCSI disk' | head
- [Thu Jun  1 14:06:48 2017] sd 17:0:1:0: [sdb] Attached SCSI disk
- [Thu Jun  1 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Mauricio Faria de Oliveira]

** Description changed:

  [Impact]
  
-  * The SES driver causes a long delay in disk discovery when 
-a large number of disks is present in the disk enclosure,
-which increases with the number of disks attached.
- 
-  * This delays the addition and visibility of the disk devices
-to userspace, which among other things causes multipath not
-to have multiple paths, actually, until the disk discovery
-eventually/finally finishes.
- 
-  * The fix significantly shortens the time taken by the SES
-driver to handle disk discovery, causing no extra delays,
-by removing a superfluous SCSI command sent to enclosure.
+  * The SES driver causes a long delay in disk discovery when
+    a large number of disks is present in the disk enclosure,
+    which increases with the number of disks attached.
+ 
+  * This delays the addition and visibility of the disk devices
+    to userspace, which among other things causes multipath not
+    to have multiple paths, actually, until the disk discovery
+    eventually/finally finishes.
+ 
+  * The fix significantly shortens the time taken by the SES
+    driver to handle disk discovery, causing no extra delays,
+    by removing a superfluous SCSI command sent to enclosure.
  
  [Test Case]
  
-  * Load the module to access the enclosure and its disks; e.g.,
- 
-$ sudo modprobe mpt3sas
- 
-  * Notice the interval between the discovery of each disk; e.g., dmesg
- 
-$ dmesg -T | grep 'Attached SCSI disk' | tail -n2
-[Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
-[Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
- 
-  * The interval should be in the same second or so range with the fix.
+  * Load the module to access the enclosure and its disks; e.g.,
+ 
+    $ sudo modprobe mpt3sas
+ 
+  * Notice the interval between the discovery of each disk; e.g., dmesg
+ 
+    $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
+    [Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
+    [Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
+ 
+  * The interval should be in the same second or so range with the fix.
+ 
+    $ dmesg -T | grep 'Attached SCSI disk' | tail -n2
+    [Wed Jun  7 13:11:59 2017] sd 18:0:176:0: [sdly] Attached SCSI disk
+    [Wed Jun  7 13:11:59 2017] sd 18:0:175:0: [sdlx] Attached SCSI disk
  
  [Regression Potential]
  
-  * The power status of the disks in the enclosure is no longer
-checked during probe time.  However, the patch demonstrates that
-initial value was never used in any way.  So, little regression
-potential.
- 
-  * Nonetheless, users of SES enclosures which verify the power status
-of disks in the enclosure might _theoretically_ see a problem, iff
-the fix has a problem (which has not been found yet).
+  * The power status of the disks in the enclosure is no longer
+    checked during probe time.  However, the patch demonstrates that
+    initial value was never used in any way.  So, little regression
+    potential.
+ 
+  * Nonetheless, users of SES enclosures which verify the power status
+    of disks in the enclosure might _theoretically_ see a problem, iff
+    the fix has a problem (which has not been found yet).
  
  [Other Info]
-  
-  * None at this time.
+ 
+  * None at this time.
  
   State: Open by: nguyenp on 31 May 2017 15:46:14 
  
   Product Name  : OpenPOWER Firmware
   Product Version   : open-power-SMC-P8DTU-V2.00.GA2-20170126-prod
   Product Extra :  op-build-3782262
   Product Extra :  hostboot-7fdfb37
   Product Extra :  occ-e6e194f
   Product Extra :  skiboot-5.4.2
   Product Extra :  linux-4.4.24-openpower1-9641b3a
   Product Extra :  petitboot-v1.4.0-2f8598b
   Product Extra :  p8dtu-xml-9a8fee2
  
  Cable configuration:
  
  On this P8-Briggs system, I have 2 Seagate Storages running with max 
configuration. There are 84 HDDs drives in each storage. So the total drives is 
168 HDDs for both Seagate storages.
  
  I connected 2 LSI 9300-8e SAS adapters to 2 Seagate storages with
  alternate cabling for redundancy. See a Figure on the connection below:
  
  Note:  Each Seagate storage has 2 I/O moudules connection in the back.
     Both I/O modules from each Seagate does see the same set of HDDs
  
  Cable connection:
  
  SAS adapter #1:port1  ->  Seagate #1-A I/O module
     port0  --> Seagate 
#2-B I/O module
  
  SAS adapter #2:port1  >  Seagate #2-A I/O module
     port0  --> Seagate 
#1-B I/O module
  
  Ubuntu 16.04.2:
  ===
  
  - Running with new kernel Ubuntu 4.8.0-520-generic #550~16.04.1+bz154734
  from Mauricio Faria De Oliveira.
  
  Problem Description:
  
  In this Briggs system, I'm running with new Ubuntu 

[Kernel-packages] [Bug 1696445] Re: OpenPower: Some multipaths temporarily have only a single path

[Mauricio Faria de Oliveira]

** Description changed:

+ [Impact]
+ 
+  * The SES driver causes a long delay in disk discovery when 
+a large number of disks is present in the disk enclosure,
+which increases with the number of disks attached.
+ 
+  * This delays the addition and visibility of the disk devices
+to userspace, which among other things causes multipath not
+to have multiple paths, actually, until the disk discovery
+eventually/finally finishes.
+ 
+  * The fix significantly shortens the time taken by the SES
+driver to handle disk discovery, causing no extra delays,
+by removing a superfluous SCSI command sent to enclosure.
+ 
+ [Test Case]
+ 
+  * Load the module to access the enclosure and its disks; e.g.,
+ 
+$ sudo modprobe mpt3sas
+ 
+  * Notice the interval between the discovery of each disk; e.g., dmesg
+ 
+$ dmesg -T | grep 'Attached SCSI disk' | tail -n2
+[Thu Jun 1 14:18:30 2017] sd 17:0:100:0: [sdcr] Attached SCSI disk
+[Thu Jun 1 14:18:35 2017] sd 17:0:101:0: [sdcs] Attached SCSI disk
+ 
+  * The interval should be in the same second or so range with the fix.
+ 
+ [Regression Potential]
+ 
+  * The power status of the disks in the enclosure is no longer
+checked during probe time.  However, the patch demonstrates that
+initial value was never used in any way.  So, little regression
+potential.
+ 
+  * Nonetheless, users of SES enclosures which verify the power status
+of disks in the enclosure might _theoretically_ see a problem, iff
+the fix has a problem (which has not been found yet).
+ 
+ [Other Info]
+  
+  * None at this time.
+ 
   State: Open by: nguyenp on 31 May 2017 15:46:14 
  
-  Product Name  : OpenPOWER Firmware
-  Product Version   : open-power-SMC-P8DTU-V2.00.GA2-20170126-prod
-  Product Extra :  op-build-3782262
-  Product Extra :  hostboot-7fdfb37
-  Product Extra :  occ-e6e194f
-  Product Extra :  skiboot-5.4.2
-  Product Extra :  linux-4.4.24-openpower1-9641b3a
-  Product Extra :  petitboot-v1.4.0-2f8598b
-  Product Extra :  p8dtu-xml-9a8fee2
+  Product Name  : OpenPOWER Firmware
+  Product Version   : open-power-SMC-P8DTU-V2.00.GA2-20170126-prod
+  Product Extra :  op-build-3782262
+  Product Extra :  hostboot-7fdfb37
+  Product Extra :  occ-e6e194f
+  Product Extra :  skiboot-5.4.2
+  Product Extra :  linux-4.4.24-openpower1-9641b3a
+  Product Extra :  petitboot-v1.4.0-2f8598b
+  Product Extra :  p8dtu-xml-9a8fee2
  
  Cable configuration:
  
  On this P8-Briggs system, I have 2 Seagate Storages running with max 
configuration. There are 84 HDDs drives in each storage. So the total drives is 
168 HDDs for both Seagate storages.
  
  I connected 2 LSI 9300-8e SAS adapters to 2 Seagate storages with
  alternate cabling for redundancy. See a Figure on the connection below:
  
- Note:  Each Seagate storage has 2 I/O moudules connection in the back. 
-Both I/O modules from each Seagate does see the same set of HDDs
+ Note:  Each Seagate storage has 2 I/O moudules connection in the back.
+    Both I/O modules from each Seagate does see the same set of HDDs
  
  Cable connection:
  
  SAS adapter #1:port1  ->  Seagate #1-A I/O module
-port0  --> Seagate 
#2-B I/O module
- 
- 
+    port0  --> Seagate 
#2-B I/O module
+ 
  SAS adapter #2:port1  >  Seagate #2-A I/O module
-port0  --> Seagate 
#1-B I/O module
+    port0  --> Seagate 
#1-B I/O module
  
  Ubuntu 16.04.2:
  ===
  
  - Running with new kernel Ubuntu 4.8.0-520-generic #550~16.04.1+bz154734
  from Mauricio Faria De Oliveira.
  
  Problem Description:
  
  In this Briggs system, I'm running with new Ubuntu 4.8.0-520-generic 
#550~16.04.1+bz154734 that has fix for Multipath problem. Mauricio helped to 
patch the system with this kernel last week to fix the multipath io_setup 
failed problem in LTCBug154734.
  
  This week, I went ahead and scaled up my test configuration to max
  configuration 2x5U84_Enclosures,_MaxCfg_168HDDs. This time, it hit a
  different issue. The issue is that some multipaths only have a single
  path and no redundancy. Others have multiple paths and redundancy.
  
  == Comment: #13 - Paul Nguyen - 2017-06-01 15:19:58 ==
  
  -  I agreed with Mauricio that this problem is a timing problem.
  
  - I re-ran the test and noticed that it took more than 50 minutes after
  system reboot to discover all disks and to build Multipaths correctly.
  
  - So for it to take this long, it's going to be a problem.
  
  - I have gathered all logs and attaching to