[Bug 460906] Re: disk/by-uuid/foo symlink points to snapshot rather than the origin

2012-08-21 Thread Steve Fisher
Possibly users should be warned in some prominent location that if they're mounting by UUID then leaving an LVM snapshot in place could result in that being mounted on reboot. Changing the UUID of the 'real' disk during live operation simply because it's been snapshotted sounds reasonably insane

[Bug 1030667] [NEW] LVM2 regex exclusion filtering applies incorrectly in some cases

2012-07-29 Thread Steve Fisher
Public bug reported: lvm.conf: If I add a|.*| into the regex array for the filter, it ignores my 'sd[b-z]' exclusions and any other items (eg loop, ram) after the [b-z]. It seems to be an issue with the use of square brackets inside the regex in certain ways. Eg if I use the line: filter = [

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-27 Thread Steve Fisher
The root cause was in fact the lvm.conf filter, but explicitly not for the reason you'd think. The issue is that if I added a|.*| into regex array, it was ignoring my 'sd[b-z]', loop and ram exclusions, both singly and in combination. It seems to be an obscure issue with the use of square

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-27 Thread Steve Fisher
The root cause was in fact the lvm.conf filter, but explicitly not for the reason you'd think. The issue is that if I added a|.*| into regex array, it was ignoring my 'sd[b-z]', loop and ram exclusions, both singly and in combination. It seems to be an obscure issue with the use of square

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-24 Thread Steve Fisher
Output from before and after commands is attached. I'm pretty sure you're right about the LVM device filter; I figured that setting the scsi-timeout to 90 seconds (being way longer than the TUR path checker, which is scheduled every 20 seconds) should be enough to handle the device failover.

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-24 Thread Steve Fisher
Output from before and after commands is attached. I'm pretty sure you're right about the LVM device filter; I figured that setting the scsi-timeout to 90 seconds (being way longer than the TUR path checker, which is scheduled every 20 seconds) should be enough to handle the device failover.

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-23 Thread Steve Fisher
It looks like all the /dev/mapper/mpathX targets, and the remaining 'sd' unique paths (with one fabric disabled) were all readable after the path shutdown, but the underlying LVs were somehow still deactivated when the paths disappeared. root@hostname-03:/srv/mysql# dmesg snip [1039812.298161]

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-23 Thread Steve Fisher
It looks like all the /dev/mapper/mpathX targets, and the remaining 'sd' unique paths (with one fabric disabled) were all readable after the path shutdown, but the underlying LVs were somehow still deactivated when the paths disappeared. root@hostname-03:/srv/mysql# dmesg snip [1039812.298161]

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-16 Thread Steve Fisher
To date, the issue hasn't been observed on the two physical hosts we have running 10.04.1 LTS with the same multipath-tools version, which certainly raises a flag. They are being used as mission critical / production database servers so I'm scheduling a window in which we will be able to confirm

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-16 Thread Steve Fisher
To date, the issue hasn't been observed on the two physical hosts we have running 10.04.1 LTS with the same multipath-tools version, which certainly raises a flag. They are being used as mission critical / production database servers so I'm scheduling a window in which we will be able to confirm

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
** Attachment removed: strace_mpathd.log https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3218020/+files/strace_mpathd.log ** Attachment added: strace_mpathd.log

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
-- lvm.conf blacklist should be fine, searches /dev but filters all /dev/sd.* I'll attach those files momentarily. I didn't see any kpartx processes running at all; there wasn't anything in an uninterruptable sleep state ('D') either. Listing all kernel and some additional processes here

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
Storage devices: $ sudo lsscsi [0:0:0:0]cd/dvd TSSTcorp CDDVDW TS-L633F IT03 /dev/sr0 [4:2:0:0]diskIBM ServeRAID M5015 2.12 /dev/sda [5:0:0:0]diskHITACHI OPEN-V 6008 /dev/sdb [5:0:0:1]diskHITACHI OPEN-V 6008 /dev/sdc [5:0:0:2]disk

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
I note this segfault has appeared in dmesg on the last couple of boots. [ 304.214560] multipathd[3083]: segfault at a ip 7f5eb838798a sp 7fffb225ebb0 error 4 in libc-2.11.1.so[7f5eb831+17a000] After adjusting the blacklist, updating initrd and rebooting, the same behaviour is

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
** Attachment added: strace_mpathd_110712.log https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3219323/+files/strace_mpathd_110712.log -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
** Attachment added: multipathd_stdout_110712.log https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3219324/+files/multipathd_stdout_110712.log -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
** Attachment removed: strace_mpathd.log https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3218020/+files/strace_mpathd.log ** Attachment added: strace_mpathd.log

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
Storage devices: $ sudo lsscsi [0:0:0:0]cd/dvd TSSTcorp CDDVDW TS-L633F IT03 /dev/sr0 [4:2:0:0]diskIBM ServeRAID M5015 2.12 /dev/sda [5:0:0:0]diskHITACHI OPEN-V 6008 /dev/sdb [5:0:0:1]diskHITACHI OPEN-V 6008 /dev/sdc [5:0:0:2]disk

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
I note this segfault has appeared in dmesg on the last couple of boots. [ 304.214560] multipathd[3083]: segfault at a ip 7f5eb838798a sp 7fffb225ebb0 error 4 in libc-2.11.1.so[7f5eb831+17a000] After adjusting the blacklist, updating initrd and rebooting, the same behaviour is

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
** Attachment added: strace_mpathd_110712.log https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3219323/+files/strace_mpathd_110712.log -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-10 Thread Steve Fisher
** Attachment added: multipathd_stdout_110712.log https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3219324/+files/multipathd_stdout_110712.log -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
Hi Peter, Thanks for picking this up. Note: I ran an 'update-initramfs -u -k all' and rebooted just for good measure before proceeding. There's some output regarding a missing firmware file, I'm not sure it's relevant: root@rgrprod-pmdh-proc-03:/etc# update-initramfs -u -k all

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
note: multipath_stderr.log exists, but is empty. ** Attachment added: multipathd_stdout.log https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3218021/+files/multipathd_stdout.log -- You received this bug notification because you are a member of Ubuntu Server

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
user@hostname:~$ sudo dmsetup table mpath2: 0 8388608 multipath 1 queue_if_no_path 0 1 1 round-robin 0 4 1 8:48 1000 8:96 1000 8:144 1000 8:192 1000 mpath1: 0 8388608 multipath 1 queue_if_no_path 0 1 1 round-robin 0 4 1 8:128 1000 8:32 1000 8:80 1000 8:176 1000 mpath0: 0 1048576000 multipath 1

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
** Attachment added: multipath.conf https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3218022/+files/multipath.conf -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to multipath-tools in Ubuntu.

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
** Attachment added: lvm.conf https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3218023/+files/lvm.conf -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to multipath-tools in Ubuntu.

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
note: /dev/sda is a raid volume used for the root vg; the filter is actually r|/dev/sd[b-z]| -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to multipath-tools in Ubuntu. https://bugs.launchpad.net/bugs/1020436 Title: Cannot read

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
Hi Peter, Thanks for picking this up. Note: I ran an 'update-initramfs -u -k all' and rebooted just for good measure before proceeding. There's some output regarding a missing firmware file, I'm not sure it's relevant: root@rgrprod-pmdh-proc-03:/etc# update-initramfs -u -k all

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
note: multipath_stderr.log exists, but is empty. ** Attachment added: multipathd_stdout.log https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3218021/+files/multipathd_stdout.log -- You received this bug notification because you are a member of Ubuntu Bugs,

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
user@hostname:~$ sudo dmsetup table mpath2: 0 8388608 multipath 1 queue_if_no_path 0 1 1 round-robin 0 4 1 8:48 1000 8:96 1000 8:144 1000 8:192 1000 mpath1: 0 8388608 multipath 1 queue_if_no_path 0 1 1 round-robin 0 4 1 8:128 1000 8:32 1000 8:80 1000 8:176 1000 mpath0: 0 1048576000 multipath 1

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
-- lvm.conf blacklist should be fine, searches /dev but filters all /dev/sd.* I'll attach those files momentarily. I didn't see any kpartx processes running at all; there wasn't anything in an uninterruptable sleep state ('D') either. Listing all kernel and some additional processes here

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
** Attachment added: multipath.conf https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3218022/+files/multipath.conf -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
** Attachment added: lvm.conf https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1020436/+attachment/3218023/+files/lvm.conf -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1020436

[Bug 1020436] Re: Cannot read superblock after FC multipath failover

2012-07-09 Thread Steve Fisher
note: /dev/sda is a raid volume used for the root vg; the filter is actually r|/dev/sd[b-z]| -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1020436 Title: Cannot read superblock after FC multipath

[Bug 1020436] Re: Cannot read superblock after fibre path failover

2012-07-08 Thread Steve Fisher
** Summary changed: - Multiple filesystems cannot read superblock after fibre path failover + Cannot read superblock after fibre path failover ** Summary changed: - Cannot read superblock after fibre path failover + Cannot read superblock after FC multipath failover -- You received this bug

[Bug 1020436] Re: Cannot read superblock after fibre path failover

2012-07-08 Thread Steve Fisher
** Summary changed: - Multiple filesystems cannot read superblock after fibre path failover + Cannot read superblock after fibre path failover ** Summary changed: - Cannot read superblock after fibre path failover + Cannot read superblock after FC multipath failover -- You received this bug

[Bug 1020436] [NEW] Multiple filesystems cannot read superblock after fibre path failover

2012-07-03 Thread Steve Fisher
Public bug reported: This might not be multipath-tools, but it's the likeliest candidate. If fibre channel connectivity is disrupted on an active path, I can see the multipath daemon rejuggling active paths as expected. However instead of utilising a new path, the server continues trying active

[Bug 1020436] Re: Multiple filesystems cannot read superblock after fibre path failover

2012-07-03 Thread Steve Fisher
** Description changed: This might not be multipath-tools, but it's the likeliest candidate. If fibre channel connectivity is disrupted on an active path, I can see the multipath daemon rejuggling active paths as expected. However instead of utilising a new path, the server continues

[Bug 1020436] [NEW] Multiple filesystems cannot read superblock after fibre path failover

2012-07-03 Thread Steve Fisher
Public bug reported: This might not be multipath-tools, but it's the likeliest candidate. If fibre channel connectivity is disrupted on an active path, I can see the multipath daemon rejuggling active paths as expected. However instead of utilising a new path, the server continues trying active

[Bug 1020436] Re: Multiple filesystems cannot read superblock after fibre path failover

2012-07-03 Thread Steve Fisher
** Description changed: This might not be multipath-tools, but it's the likeliest candidate. If fibre channel connectivity is disrupted on an active path, I can see the multipath daemon rejuggling active paths as expected. However instead of utilising a new path, the server continues