This is a multi-part MIME message. Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
We are doing relocation testing under z/VM 6.3. We test a SLES 11SP2 and o= racle 10.2. We bring up on both z/vm LPARS in standalone way and works fine= on both LPARS. When we begin relocation from any of the LPAR, immediately = the SAN file system goes in read only mode. I'm attaching part of the log a= t the moment of failure. The SAN is a XIV We are running z/VM 6.3. Thanks for your help Victor Echavarry System Programmer, EVERTEC LLC WARNING: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please delete it immedi= ately. Please note that any views or opinions presented in this email are solely t= hose of the author and do not necessarily represent those of EVERTEC, Inc. or its affiliates. Finally, the integrity and security of this message cannot be guaranteed on the Internet, and as such EVERTEC, Inc. and its affiliates ac= cept no liability for any damage caused by any virus transmitted by this email. ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 ---------------------------------------------------------------------- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Jan 20 17:19:02 L209T kernel: qeth.3acf0c: 0.0.0360: The qeth device driver failed to recover an error on the device Jan 20 17:19:02 L209T kernel: qeth: irb 00000000: 00 c2 40 17 7e 05 10 38 0e 02 00 00 00 80 00 00 ..@.~..8........ Jan 20 17:19:02 L209T kernel: qeth: irb 00000010: 01 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jan 20 17:19:02 L209T kernel: qeth: sense data 00000000: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jan 20 17:19:02 L209T kernel: qeth: sense data 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jan 20 17:19:02 L209T kernel: qeth.3acf0c: 0.0.0360: The qeth device driver failed to recover an error on the device Jan 20 17:19:02 L209T kernel: qeth: irb 00000000: 00 c2 40 17 7e 05 10 38 0e 02 00 00 00 80 00 00 ..@.~..8........ Jan 20 17:19:02 L209T kernel: qeth: irb 00000010: 01 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jan 20 17:19:02 L209T kernel: qeth: irb 00000020: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jan 20 17:19:02 L209T kernel: qeth: irb 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jan 20 17:19:02 L209T kernel: zfcp.e78dec: 0.0.7548: A QDIO problem occurred Jan 20 17:19:02 L209T kernel: zfcp.e78dec: 0.0.7508: A QDIO problem occurred Jan 20 17:19:02 L209T kernel: qeth.fd0b7c: 0.0.0360: A recovery process has been started for the device Jan 20 17:19:02 L209T kernel: qdio: 0.0.0362 OSA on SC 2 using AI:1 QEBSM:0 PCI:1 TDD:1 SIGA:RW A Jan 20 17:19:02 L209T kernel: qdio: 0.0.7508 ZFCP on SC 15 using AI:1 QEBSM:1 PCI:1 TDD:1 SIGA: W A Jan 20 17:19:02 L209T kernel: qdio: 0.0.7548 ZFCP on SC 16 using AI:1 QEBSM:1 PCI:1 TDD:1 SIGA: W A Jan 20 17:19:02 L209T kernel: qeth.cc0c57: 0.0.0360: MAC address 02:78:c1:02:09:00 successfully registered on device eth0 Jan 20 17:19:02 L209T kernel: qeth.736dae: 0.0.0360: Device is a Guest LAN QDIO card (level: V630) Jan 20 17:19:02 L209T kernel: with link type GuestLAN QDIO (portname: ) Jan 20 17:19:02 L209T kernel: qeth.bad88b: 0.0.0360: Device successfully recovered! Jan 20 17:19:07 L209T multipathd: 8:0: mark as failed Jan 20 17:19:07 L209T multipathd: 20017380009c51b8f: remaining active paths: 5 Jan 20 17:19:07 L209T multipathd: 8:16: mark as failed Jan 20 17:19:07 L209T multipathd: 20017380009c51b90: remaining active paths: 5 Jan 20 17:19:07 L209T multipathd: 8:64: mark as failed Jan 20 17:19:07 L209T multipathd: 20017380009c51cfb: remaining active paths: 5 Jan 20 17:19:07 L209T multipathd: 20017380009c51b90: sdb - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: 20017380009c51b8f: sda - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: 20017380009c51cfa: sdc - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: checker failed path 8:32 in map 20017380009c51cfa Jan 20 17:19:07 L209T multipathd: 20017380009c51cfa: remaining active paths: 5 Jan 20 17:19:07 L209T multipathd: 20017380009c51cf9: sdd - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: checker failed path 8:48 in map 20017380009c51cf9 Jan 20 17:19:07 L209T multipathd: 20017380009c51cf9: remaining active paths: 5 Jan 20 17:19:07 L209T multipathd: 20017380009c51cfb: sde - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: 20017380009c51b8f: sdf - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: checker failed path 8:80 in map 20017380009c51b8f Jan 20 17:19:07 L209T multipathd: 20017380009c51b8f: remaining active paths: 4 Jan 20 17:19:07 L209T multipathd: 20017380009c51b90: sdg - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: checker failed path 8:96 in map 20017380009c51b90 Jan 20 17:19:07 L209T multipathd: 20017380009c51b90: remaining active paths: 4 Jan 20 17:19:07 L209T multipathd: 20017380009c51cfa: sdh - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: checker failed path 8:112 in map 20017380009c51cfa Jan 20 17:19:07 L209T multipathd: 20017380009c51cfa: remaining active paths: 4 Jan 20 17:19:07 L209T multipathd: 20017380009c51cf9: sdi - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: checker failed path 8:128 in map 20017380009c51cf9 Jan 20 17:19:07 L209T multipathd: 20017380009c51cf9: remaining active paths: 4 Jan 20 17:19:07 L209T multipathd: 20017380009c51cfb: sdj - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: checker failed path 8:144 in map 20017380009c51cfb Jan 20 17:19:07 L209T multipathd: 20017380009c51cfb: remaining active paths: 4 Jan 20 17:19:07 L209T multipathd: 20017380009c51b8f: sdk - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: checker failed path 8:160 in map 20017380009c51b8f Jan 20 17:19:07 L209T multipathd: 20017380009c51b8f: remaining active paths: 3 Jan 20 17:19:07 L209T multipathd: 20017380009c51b90: sdl - directio checker reports path is down Jan 20 17:19:07 L209T multipathd: checker failed path 8:176 in map 20017380009c51b90 Jan 20 17:19:07 L209T multipathd: 20017380009c51b90: remaining active paths: 3 Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] CDB: Write(10): 2a 00 03 50 ec f0 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sda, sector 55635184 Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] CDB: Write(10): 2a 00 03 f1 8d 60 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sda, sector 66162016 Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] CDB: Write(10): 2a 00 04 43 c1 e8 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sda, sector 71549416 Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] CDB: Write(10): 2a 00 04 15 71 d0 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sda, sector 68514256 Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] CDB: Write(10): 2a 00 03 d0 19 c8 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sda, sector 63969736 Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] CDB: Write(10): 2a 00 02 e4 02 48 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sda, sector 48497224 Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] CDB: Write(10): 2a 00 05 1b 92 18 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sda, sector 85692952 Jan 20 17:19:07 L209T kernel: sd 0:0:0:2: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:2: [sdb] CDB: Write(10): 2a 00 03 d3 bb b8 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdb, sector 64207800 Jan 20 17:19:07 L209T kernel: sd 0:0:0:2: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:2: [sdb] CDB: Write(10): 2a 00 04 fc 11 90 00 00 18 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdb, sector 83628432 Jan 20 17:19:07 L209T kernel: sd 0:0:0:2: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:2: [sdb] CDB: Write(10): 2a 00 04 ff 86 d8 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdb, sector 83855064 Jan 20 17:19:07 L209T kernel: sd 0:0:0:5: [sde] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:5: [sde] CDB: Write(10): 2a 00 05 a7 c1 e8 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sde, sector 94880232 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:0. Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:16. Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:64. Jan 20 17:19:07 L209T kernel: sd 0:0:0:2: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:2: [sdb] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdb, sector 0 Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:1: [sda] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sda, sector 0 Jan 20 17:19:07 L209T kernel: sd 0:0:0:3: [sdc] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:3: [sdc] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdc, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:32. Jan 20 17:19:07 L209T kernel: sd 0:0:0:4: [sdd] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:4: [sdd] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdd, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:48. Jan 20 17:19:07 L209T kernel: sd 0:0:0:5: [sde] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:0:5: [sde] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sde, sector 0 Jan 20 17:19:07 L209T kernel: sd 0:0:1:1: [sdf] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:1:1: [sdf] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdf, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:80. Jan 20 17:19:07 L209T kernel: sd 0:0:1:2: [sdg] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:1:2: [sdg] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdg, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:96. Jan 20 17:19:07 L209T kernel: sd 0:0:1:3: [sdh] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:1:3: [sdh] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdh, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:112. Jan 20 17:19:07 L209T kernel: sd 0:0:1:4: [sdi] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:1:4: [sdi] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdi, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:128. Jan 20 17:19:07 L209T kernel: sd 0:0:1:5: [sdj] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 0:0:1:5: [sdj] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdj, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:144. Jan 20 17:19:07 L209T kernel: sd 1:0:0:1: [sdk] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:0:1: [sdk] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdk, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:160. Jan 20 17:19:07 L209T kernel: sd 1:0:0:2: [sdl] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:0:2: [sdl] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdl, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:176. Jan 20 17:19:07 L209T kernel: sd 1:0:0:4: [sdn] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:0:4: [sdn] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdn, sector 0 Jan 20 17:19:07 L209T kernel: sd 1:0:1:2: [sdq] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:2: [sdq] CDB: Write(10): 2a 00 04 ff 86 d8 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdq, sector 83855064 Jan 20 17:19:07 L209T kernel: sd 1:0:1:2: [sdq] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:2: [sdq] CDB: Write(10): 2a 00 04 fc 11 90 00 00 18 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdq, sector 83628432 Jan 20 17:19:07 L209T kernel: sd 1:0:1:2: [sdq] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:2: [sdq] CDB: Write(10): 2a 00 03 d3 bb b8 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdq, sector 64207800 Jan 20 17:19:07 L209T kernel: sd 1:0:0:5: [sdo] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:0:5: [sdo] CDB: Write(10): 2a 00 05 a7 c1 e8 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdo, sector 94880232 Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] CDB: Write(10): 2a 00 05 1b 92 18 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdp, sector 85692952 Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] CDB: Write(10): 2a 00 02 e4 02 48 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdp, sector 48497224 Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] CDB: Write(10): 2a 00 03 d0 19 c8 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdp, sector 63969736 Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] CDB: Write(10): 2a 00 04 15 71 d0 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdp, sector 68514256 Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] CDB: Write(10): 2a 00 04 43 c1 e8 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdp, sector 71549416 Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] CDB: Write(10): 2a 00 03 f1 8d 60 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdp, sector 66162016 Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] CDB: Write(10): 2a 00 03 50 ec f0 00 00 20 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdp, sector 55635184 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:208. Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:0. Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:224. Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 8:240. Jan 20 17:19:07 L209T kernel: sd 1:0:0:5: [sdo] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:0:5: [sdo] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdo, sector 0 Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:1: [sdp] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdp, sector 0 Jan 20 17:19:07 L209T kernel: sd 1:0:1:2: [sdq] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:2: [sdq] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdq, sector 0 Jan 20 17:19:07 L209T kernel: sd 1:0:1:3: [sdr] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:3: [sdr] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdr, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:16. Jan 20 17:19:07 L209T kernel: sd 1:0:1:4: [sds] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:4: [sds] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sds, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:32. Jan 20 17:19:07 L209T kernel: sd 1:0:1:5: [sdt] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:1:5: [sdt] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdt, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:48. Jan 20 17:19:07 L209T kernel: sd 1:0:2:1: [sdu] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:2:1: [sdu] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdu, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:64. Jan 20 17:19:07 L209T kernel: sd 1:0:2:2: [sdv] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:2:2: [sdv] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdv, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:80. Jan 20 17:19:07 L209T kernel: sd 1:0:2:3: [sdw] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:2:3: [sdw] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdw, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:96. Jan 20 17:19:07 L209T kernel: sd 1:0:2:4: [sdx] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:2:4: [sdx] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdx, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:112. Jan 20 17:19:07 L209T kernel: sd 1:0:2:5: [sdy] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:2:5: [sdy] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdy, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:128. Jan 20 17:19:07 L209T kernel: sd 1:0:3:1: [sdz] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:3:1: [sdz] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdz, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:144. Jan 20 17:19:07 L209T kernel: sd 1:0:3:2: [sdaa] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:3:2: [sdaa] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdaa, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:160. Jan 20 17:19:07 L209T kernel: sd 1:0:3:3: [sdab] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:3:3: [sdab] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdab, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:176. Jan 20 17:19:07 L209T kernel: sd 1:0:3:4: [sdac] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:3:4: [sdac] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdac, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:192. Jan 20 17:19:07 L209T kernel: sd 1:0:3:5: [sdad] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:3:5: [sdad] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdad, sector 0 Jan 20 17:19:07 L209T kernel: device-mapper: multipath: Failing path 65:208. Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-0, sector 64207800 Jan 20 17:19:07 L209T kernel: quiet_error: 21 callbacks suppressed Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-17, logical block 8025927 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-17 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-0, sector 83628432 Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-17, logical block 10453506 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-17 Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-17, logical block 10453507 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-17 Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-17, logical block 10453508 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-17 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-0, sector 83855064 Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-17, logical block 10481835 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-17 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-4, sector 94880232 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 55635184 Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-16, logical block 6954350 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-16 Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-16, logical block 6954351 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-16 Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-16, logical block 6954352 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-16 Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-16, logical block 6954353 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-16 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 66162016 Jan 20 17:19:07 L209T kernel: Buffer I/O error on device dm-16, logical block 8270204 Jan 20 17:19:07 L209T kernel: lost page write due to I/O error on dm-16 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 71549416 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 68514256 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 63969736 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 48497224 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 85692952 Jan 20 17:19:07 L209T kernel: sd 1:0:0:3: [sdm] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK Jan 20 17:19:07 L209T kernel: sd 1:0:0:3: [sdm] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00 Jan 20 17:19:07 L209T kernel: end_request: recoverable transport error, dev sdm, sector 0 Jan 20 17:19:07 L209T kernel: JBD: Detected IO errors while flushing file data on dm-16 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-0, sector 65093560 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-0, sector 83717584 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 17220032 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49091568 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49091600 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49091640 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49091928 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 55649656 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 66175832 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 68528072 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 63983552 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 50944 Jan 20 17:19:07 L209T kernel: Aborting journal on device dm-16. Jan 20 17:19:07 L209T kernel: EXT3-fs (dm-16): error in ext3_orphan_add: Journal has aborted Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-0, sector 66491832 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-0, sector 83731288 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 55664176 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 66189648 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 68541888 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 63997368 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 12744 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 384 Jan 20 17:19:07 L209T kernel: JBD: I/O error detected when updating journal superblock for dm-16. Jan 20 17:19:07 L209T kernel: EXT3-fs (dm-16): I/O error while writing superblock Jan 20 17:19:07 L209T kernel: EXT3-fs (dm-16): error in ext3_write_begin: Journal has aborted Jan 20 17:19:07 L209T kernel: JBD: Detected IO errors while flushing file data on dm-16 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 384 Jan 20 17:19:07 L209T kernel: EXT3-fs (dm-16): I/O error while writing superblock Jan 20 17:19:07 L209T kernel: EXT3-fs (dm-16): error: ext3_journal_start_sb: Detected aborted journal Jan 20 17:19:07 L209T kernel: EXT3-fs (dm-16): error: remounting filesystem read-only Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49473912 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474112 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474136 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474176 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474280 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474320 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474384 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474416 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474520 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474552 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474584 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474632 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474672 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474776 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474824 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474912 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474944 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49474424 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49497008 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49497064 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49497080 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49497096 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49092440 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49088056 Jan 20 17:19:07 L209T kernel: end_request: I/O error, dev dm-1, sector 49497080
