** Summary changed: - [All releases] Suspending with rootfs on USB, causes silent corruptions, kernel panic on mount attempt, data loss and leaving system unbootable. + [All releases] Suspend/Resume with rootfs on USB, causes filesystem corruptions and kernel panic on mount attempt, leaving system unbootable with data lost.
** Description changed: - Suspend of root filesystem on USB, causes FS corruptions which crashes - kernel on mount attempt and making systen unbootable. + Suspend of root filesystem on USB, causes FS corruptions which crashes kernel + on mount attempt and making systen unbootable. - Affects Karmic, Lucid, Maverick and Natty. + Affects (at least) RedHat6.0+, Fedora13+ and Ubuntu10+. - Reconstruction: + Reconstruction (Full): --------------- - a) Install any Ubuntu release on USB drive. - b) Boot into user session (preferably with ecryptfs enabled, which raises probably to almost always) + a) Install any RedHat/Fedora/Ubuntu release on USB drive. + b) Boot into user session c) Suspend - d) Variant1: Resume -> Observer FS inconsistence and inability to continue operation with rootfs. - Afer hard reset (system unable to reboot/shutdown normally with readonly rootfs) the system is unable to boot due silently corrupted FS. - Variant2: Eject device -> Try to mount it on any Ubuntu release -> Observe Kernel panic. - Variant3: Hard reset the system (Before performing resume) -> Observe system inability to boot due to corrupted FS. + d) Resume + e) Repeat c,d (up to 4 times) until kernel informs about filesystem corruption. + f) Reset the system -> Observe system inability to boot and kernel panic due to + corrupted FS. + + Reconstruction (Short): + --------------- + a) Install any RedHat/Fedora/Ubuntu release on USB drive. + b) Boot into user session + c) Suspend + d) Hard reset the system (Before performing resume) -> Observe system inability + to boot and kernel panic due to corrupted FS. Actually report describes two severe issues: -------------------------------------------- - a) FS corruption during suspend + a) FS corruption during suspend/resume b) Kernel panic when trying to mount affected FS. Facts: ------ *) Happens on systems with bootdevice on USB. - *) When system returns from suspend, there is a chance of ~25% for rootfs on USB to be remounted as readonly. Due to short inability to read from USB device (device settle delay required) - *) Attempt to start affected FS will result an useless initramfs prompt without ability to solve the issue locally. - *) To fix the issue additional Linux system required (with auto-mount de-activated!). + *) When system returns from suspend, there is a chance of ~25% for rootfs on + USB to be remounted as readonly. Due to short inability to read from USB device + (device settle delay required) + *) Attempt to start affected FS will result an useless initramfs prompt without + ability to solve the issue locally. + *) To fix the issue additional Linux system required (with auto-mount + de-activated!). *) An attempt to mount affected EXT4 will results a crashed provided below. - Corruptions are silent, FS marked as clean, that's what causes Kernel panic when it tries to mount corrupted FS marked as clean. + Corruptions are silent, FS marked as clean, that's what causes Kernel panic + when it tries to mount corrupted FS marked as clean. *) Issue caused probably by partially synced disk cache. - *) All tested Kernels affected (Lucid, Maverick, Natty, Natty-Mainline-Daily, Fedora14, Redhat6). - *) Validated with: Vanilla Lucid, Maverick, Natty, Fedora14 and RedHat6. - *) Kernels above 2.6.38 manages to mark FS as dirty during failed mount attempt, so virtually corruption got fixed after 2 reboots but loss is there. - *) Bug affects all filesystems, but severe for EXT4(data loss while fixable with another system without automount) and BTRFS (unfixable due to failed mounttime fixes and there is no way to fix manually). - - As immediate solution I propose: - -------------------------------- - 1) Fix suspend/resume whith bootdevice on USB. - 2) Migration of filesystem recovery to initramfs-tools. Will help system to recover before corrupted rootfs mounted. - - - Below log of attempt to mount corrupted FS: - ------------------------------------------------------------------------------------- - [225221.546381] usb-storage: device found at 22 - [225221.546386] usb-storage: waiting for device to settle before scanning - [225226.551130] usb-storage: device scan complete - [225226.555578] scsi 11:0:0:0: Direct-Access Generic Flash HS-CF 4.44 PQ: 0 ANSI: 0 - [225226.558965] scsi 11:0:0:1: Direct-Access Generic Flash HS-COMBO 4.44 PQ: 0 ANSI: 0 - [225226.560622] sd 11:0:0:0: Attached scsi generic sg2 type 0 - [225226.562393] sd 11:0:0:1: Attached scsi generic sg3 type 0 - [225226.622322] sd 11:0:0:0: [sdc] Attached SCSI removable disk - [225226.626721] sd 11:0:0:1: [sdd] Attached SCSI removable disk - [225233.188100] usb 1-2: new high speed USB device using ehci_hcd and address 5 - [225233.323593] usb 1-2: configuration #1 chosen from 1 choice - [225233.324302] scsi12 : SCSI emulation for USB Mass Storage devices - [225233.324731] usb-storage: device found at 5 - [225233.324741] usb-storage: waiting for device to settle before scanning - [225238.320747] usb-storage: device scan complete - [225238.322831] scsi 12:0:0:0: Direct-Access USB Mass Storage Device PQ: 0 ANSI: 0 CCS - [225238.324132] sd 12:0:0:0: Attached scsi generic sg4 type 0 - [225238.325568] sd 12:0:0:0: [sde] 15954944 512-byte logical blocks: (8.16 GB/7.60 GiB) - [225238.326653] sd 12:0:0:0: [sde] Write Protect is off - [225238.326661] sd 12:0:0:0: [sde] Mode Sense: 03 00 00 00 - [225238.326667] sd 12:0:0:0: [sde] Assuming drive cache: write through - [225238.334023] sd 12:0:0:0: [sde] Assuming drive cache: write through - [225238.334044] sde: sde1 - [225238.338676] sd 12:0:0:0: [sde] Assuming drive cache: write through - [225238.338693] sd 12:0:0:0: [sde] Attached SCSI removable disk - [225252.842616] EXT4-fs warning (device sde1): ext4_clear_journal_err: Filesystem error recorded from previous mount: IO failure - [225252.842629] EXT4-fs warning (device sde1): ext4_clear_journal_err: Marking fs in need of filesystem check. - [225252.842674] BUG: unable to handle kernel paging request at 02add000 - [225252.842684] IP: [<c035c35a>] __percpu_counter_sum+0x2a/0x70 - [225252.842701] *pde = 00000000 - [225252.842707] Oops: 0000 [#1] SMP - [225252.842714] last sysfs file: /sys/devices/pci0000:00/0000:00:0b.0/host1/target1:0:0/1:0:0:0/block/sdb/sdb3/stat - [225252.842722] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs reiserfs hidp aes_i586 aes_generic binfmt_misc ppdev rfcomm sco bridge stp bnep l2cap ip6table_filter ip6_tables iptable_filter ip_tables x_tables dm_crypt snd_hda_codec_nvhdmi snd_hda_codec_realtek nfsd exportfs snd_hda_intel nfs snd_hda_codec lockd snd_hwdep nfs_acl auth_rpcgss snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss lm90 snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq vga16fb sunrpc coretemp vgastate nvidia(P) snd_timer snd_seq_device snd agpgart soundcore btusb i2c_nforce2 gspca_zc3xx lp gspca_main snd_page_alloc videodev ftdi_sio shpchp parport v4l1_compat bluetooth usbserial usb_storage raid10 raid456 async_raid6_recov async_pq raid6_pq async_xor async_memcpy async_tx raid0 multipath linear dm_raid45 xor usbhid hid fbcon tileblit font bitblit softcursor vesafb raid1 ahci forcedeth ramzswap xvmalloc lzo_decompress lzo_compress - [225252.842886] - [225252.842894] Pid: 14666, comm: mount Tainted: P (2.6.32-28-generic #55-Ubuntu) To Be Filled By O.E.M. - [225252.842901] EIP: 0060:[<c035c35a>] EFLAGS: 00010297 CPU: 0 - [225252.842909] EIP is at __percpu_counter_sum+0x2a/0x70 - [225252.842914] EAX: 00000000 EBX: 00000000 ECX: 02add000 EDX: 00000000 - [225252.842920] ESI: 00000000 EDI: ea39ec94 EBP: f0085d78 ESP: f0085d6c - [225252.842926] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 - [225252.842933] Process mount (pid: 14666, ti=f0084000 task=f02d5940 task.ti=f0084000) - [225252.842938] Stack: - [225252.842941] f5c4c000 03477fda 00000000 f0085dac c02a3856 e6984400 f0085d94 c058bf3e - [225252.842955] <0> 00000001 e6984400 03476c6a 00000000 f6e022c0 f5c4c000 dd309000 e6984400 - [225252.842971] <0> f0085de4 c02a3a33 f5c4c000 c05a978d c06d59dc c06e4685 dd309000 f0085de4 - [225252.842987] Call Trace: - [225252.842998] [<c02a3856>] ? ext4_commit_super+0xd6/0x1e0 - [225252.843008] [<c058bf3e>] ? printk+0x1d/0x1f - [225252.843017] [<c02a3a33>] ? ext4_clear_journal_err+0xa3/0xd0 - [225252.843026] [<c02cce5f>] ? jbd2_journal_load+0x4f/0xb0 - [225252.843035] [<c02a5f0f>] ? ext4_load_journal+0x14f/0x2e0 - [225252.843044] [<c02a71b6>] ? ext4_fill_super+0x1116/0x1930 - [225252.843055] [<c020c172>] ? get_sb_bdev+0x162/0x1a0 - [225252.843063] [<c02a60a0>] ? ext4_fill_super+0x0/0x1930 - [225252.843072] [<c02a16f6>] ? ext4_get_sb+0x26/0x30 - [225252.843080] [<c02a60a0>] ? ext4_fill_super+0x0/0x1930 - [225252.843087] [<c020bce7>] ? vfs_kern_mount+0x67/0x170 - [225252.843097] [<c0220623>] ? get_fs_type+0x33/0xb0 - [225252.843105] [<c020be4e>] ? do_kern_mount+0x3e/0xe0 - [225252.843113] [<c0223763>] ? do_mount+0x1c3/0x200 - [225252.843120] [<c022380b>] ? sys_mount+0x6b/0xa0 - [225252.843130] [<c01033ec>] ? syscall_call+0x7/0xb - [225252.843139] [<c0590000>] ? setjmp_pre_handler+0x30/0x90 - [225252.843144] Code: 00 55 89 e5 57 89 c7 56 53 e8 23 24 23 00 8b 5f 04 b8 ff ff ff ff 8b 77 08 eb 1c 8d b6 00 00 00 00 8b 0c 85 00 3f 7a c0 8b 57 14 <8b> 14 0a 89 d1 c1 f9 1f 01 d3 11 ce 8d 48 01 a1 0c c0 59 c0 ba - [225252.843225] EIP: [<c035c35a>] __percpu_counter_sum+0x2a/0x70 SS:ESP 0068:f0085d6c - [225252.843236] CR2: 0000000002add000 - [225252.843243] ---[ end trace 2bc710c0004faec8 ]--- + *) All tested Kernels affected (Lucid, Maverick, Natty, Natty-Mainline-Daily, + Fedora14, Redhat6). + *) Validated with: Mainline, Lucid, Maverick, Natty, Fedora14 and RedHat6. + *) Kernels above 2.6.38 manages to mark FS as dirty during failed mount + attempt, so virtually corruption got fixed after 2 reboots but loss is there. + *) Bug affects all filesystems, but severe for EXT4(data loss while fixable + with another system without automount) and BTRFS (unfixable due to failed + mounttime fixes and there is no way to fix manually). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/706795 Title: [All releases] Suspend/Resume with rootfs on USB, causes filesystem corruptions and kernel panic on mount attempt, leaving system unbootable with data lost. -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
