[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
** Information type changed from Public to Public Security -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in linux package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88 EFLAGS: 00010296 [153523.896276] RAX: RBX: RCX: 0082 [153523.896276] RDX: 12ea RSI: 81a6a3e9 RDI: [153523.896276] RBP: 88a97b18 R08: 81a74110 R09: 0005 [153523.896276] R10: ffe2 R11: 88a977fe R12: 0082 [153523.896276] R13: 81835280 R14: 12ea R15: 81a74110 [153523.896276] FS: 7ff43c02b840() GS:88000b80() knlGS: [153523.896276] CS: 0010 DS: ES: CR0: 8005003b
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
Hi, I've been running the following kernel for a while and have reproduced the error in this bug with the following kernel. Linux mirror 3.15.0-031500rc2-generic #201404201435 SMP Sun Apr 20 18:36:18 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Regards Russell -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in linux package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88 EFLAGS: 00010296 [153523.896276] RAX: RBX: RCX: 0082 [153523.896276] RDX: 12ea RSI: 81a6a3e9 RDI: [153523.896276] RBP: 88a97b18 R08: 81a74110 R09: 0005 [153523.896276] R10: ffe2 R11: 88a977fe R12: 0082 [153523.896276] R13: 81835280 R14:
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
The filesystem details are as follows. This may be related to an historic bug in the filesystem that is now causing ongoing corruption as the file system was created in 2010. mr-russ@mirror:~$ sudo tune2fs -l /dev/vda1 tune2fs 1.42.9 (4-Feb-2014) Filesystem volume name: none Last mounted on: / Filesystem UUID: 8c150a35-6a37-41c4-9155-a99e5acce610 Filesystem magic number: 0xEF53 Filesystem revision #:1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options:(none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 1962240 Block count: 7863809 Reserved block count: 392018 Free blocks: 2843701 Free inodes: 1509317 First block: 0 Block size: 4096 Fragment size:4096 Reserved GDT blocks: 1022 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8176 Inode blocks per group: 511 Flex block group size:16 Filesystem created: Sat May 22 20:51:25 2010 Last mount time: Fri Apr 3 09:34:17 2015 Last write time: Fri Apr 3 09:34:16 2015 Mount count: 1 Maximum mount count: 23 Last checked: Fri Apr 3 09:33:20 2015 Check interval: 15552000 (6 months) Next check after: Wed Sep 30 08:33:20 2015 Lifetime writes: 310 GB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 28 Desired extra isize: 28 Journal inode:8 Default directory hash: half_md4 Directory Hash Seed: af2303c6-1959-47cd-9fbf-3478192a400a Journal backup: inode blocks -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in linux package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1]
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
I just upgraded my hypervisor to Ubuntu 14.04 in an attempt to resolve this issue as Simon experienced. However under heavy I/O of 'aptitude upgrade', I've had 3 more occurences of this with hours of upgrading the host to 14.04. So it appears not to be related to the hypervisor. This leads me down the path that most of the affected systems have been upgraded from 10.04 - 12.04 - 14.04. Which would point to something in the upgrade path affecting this. As Simon re-installed all of his systems and didn't have the issue anymore. I could try random kernels on machines, but some information from a kernel developer on how to debug this would be very useful. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in linux package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
Hi, I behaved lazily... and have been trying to collect more data before posting back. Sorry for the delay. 1. I can only reproduce the bug about once a week on 3.13 kernel that is running on ubuntu 14.04. 2. I was lazy and after reading patches and kernel stuff, I marked it as fixed upstream. I didn't want to go compiling my own kernel as I need all the virtio options. I have 5 virtual machines running on top of a md10 + md1 raid all put into a single lvm volume. The host is running ubuntu 12.04 with a 3.2 kernel series. The guests are all virtio and running 14.04 as they have been upgraded. Most of the 5 instances have experienced a file system read-only event. One server has experienced 3. It is my package mirror running approx. So after further investigation this may not be resolved upstream. If you can provide me with a virtio/ubuntu compatible newer kernel I can put it on a machine and try it out. But because I can't reproduce this at will it's very hard to give a reproduction case. I found http://lists.openwall.net/linux-ext4/2014/05/21/2 which indicates there is a bug somewhere if we are seeing failures where we are. The result of that thread was more debugging put into the section of code that reports this. I'm happy to track and patch any amount of debug. As the filesystem re-mounts readonly, I suspect I should move /var/log onto another filesystem to increase the chances I'll log everything. All the instances I've seen this on are upgraded from at least 12.04. Some from 10.04. So there are ext3 - ext4 - ext4+ options involved in the filesystem makeup. Reading some of the bugs on the kernel list, it feel like this might be related to that an it will be complicated. I've attached to examples from tune2fs to show that I have different features attached to the different servers that are experiencing this issue. If you can provide any debugging advise that would be good, I've not really done kernel debugging before. I done a lot in the application space so I understand all the concepts. ** Attachment added: tune2fs output from bob server. https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1298972/+attachment/442/+files/bob.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494]
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
Also the two servers are running; Linux mirror 3.13.0-36-generic #63-Ubuntu SMP Wed Sep 3 21:30:07 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux If there is any other information you want me to attach, please ask. Thanks Russell. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88 EFLAGS: 00010296 [153523.896276] RAX: RBX: RCX: 0082 [153523.896276] RDX: 12ea RSI: 81a6a3e9 RDI: [153523.896276] RBP: 88a97b18 R08: 81a74110 R09: 0005 [153523.896276] R10: ffe2 R11: 88a977fe R12: 0082 [153523.896276] R13: 81835280 R14: 12ea R15: 81a74110
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
** Attachment added: tune2fs output from mirror server. https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1298972/+attachment/443/+files/mirror.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88 EFLAGS: 00010296 [153523.896276] RAX: RBX: RCX: 0082 [153523.896276] RDX: 12ea RSI: 81a6a3e9 RDI: [153523.896276] RBP: 88a97b18 R08: 81a74110 R09: 0005 [153523.896276] R10: ffe2 R11: 88a977fe R12: 0082 [153523.896276] R13: 81835280 R14: 12ea R15: 81a74110 [153523.896276] FS: 7ff43c02b840()
Re: [Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
Hi Russell, On 10/02/2014 06:06 AM, Russell Smith wrote: I behaved lazily... and have been trying to collect more data before posting back. Sorry for the delay. 1. I can only reproduce the bug about once a week on 3.13 kernel that is running on ubuntu 14.04. 2. I was lazy and after reading patches and kernel stuff, I marked it as fixed upstream. I didn't want to go compiling my own kernel as I need all the virtio options. Ubuntu provides upstream kernels here: http://kernel.ubuntu.com/~kernel-ppa/mainline/?C=N;O=D And some documentation about it is available here: https://wiki.ubuntu.com/Kernel/MainlineBuilds I have 5 virtual machines running on top of a md10 + md1 raid all put into a single lvm volume. The host is running ubuntu 12.04 with a 3.2 kernel series. The guests are all virtio and running 14.04 as they have been upgraded. Interestingly I also had an hypervisor running 12.04 and possibly with the 3.2 or the 3.11 kernel. IIRC, the guest was running a fresh install of 14.04. Since then, I reinstalled the hypervisor with 14.04 and the guest too. I never experienced this problem after this. Most of the 5 instances have experienced a file system read-only event. One server has experienced 3. It is my package mirror running approx. IIRC, my guest was an apt-cacher-ng instance. So after further investigation this may not be resolved upstream. If you can provide me with a virtio/ubuntu compatible newer kernel I can put it on a machine and try it out. But because I can't reproduce this at will it's very hard to give a reproduction case. Indeed but since I haven't reproduce this issue once I'm afraid I can't help you here. I found http://lists.openwall.net/linux-ext4/2014/05/21/2 which indicates there is a bug somewhere if we are seeing failures where we are. The result of that thread was more debugging put into the section of code that reports this. I'm happy to track and patch any amount of debug. As the filesystem re-mounts readonly, I suspect I should move /var/log onto another filesystem to increase the chances I'll log everything. You can also send the logs to a remote syslog server. Good luck and thanks! Simon -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494]
Re: [Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
Hi Russell, On 09/28/2014 01:43 AM, Russell Smith wrote: https://www.kernel.org/pub/linux/kernel/v3.0/ChangeLog-3.14.2 contains ext4 patches related to the code in this stack dump; ad6599ab3ac98a4474544086e048ce86ec15a4d1 specifically references incorrect freeing of the function. I'm not able to reliably reproduce the stack dump and am not clear on whether the commit mentioned is the complete fix for this issue. Just to be sure I understand properly, you were able to reproduce the issue reliably when using the default kernel (3.13) but cannot reproduce now that you are on 3.14? So that's why you marked the issue as fixed upstream? Any follow-up would be appreciated. Well, I was never able to reproduce the issue even while sticking to the 3.13 kernel so thank you for you input on this. Regards, Simon -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti:
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
I see this with relative consistency on a number of 14.04 VM's. I will attempt to upgrade to 3.14 and see what happens. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Expired Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88 EFLAGS: 00010296 [153523.896276] RAX: RBX: RCX: 0082 [153523.896276] RDX: 12ea RSI: 81a6a3e9 RDI: [153523.896276] RBP: 88a97b18 R08: 81a74110 R09: 0005 [153523.896276] R10: ffe2 R11: 88a977fe R12: 0082 [153523.896276] R13: 81835280 R14: 12ea R15: 81a74110 [153523.896276] FS: 7ff43c02b840() GS:88000b80() knlGS:
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
https://www.kernel.org/pub/linux/kernel/v3.0/ChangeLog-3.14.2 contains ext4 patches related to the code in this stack dump; ad6599ab3ac98a4474544086e048ce86ec15a4d1 specifically references incorrect freeing of the function. I'm not able to reliably reproduce the stack dump and am not clear on whether the commit mentioned is the complete fix for this issue. Any follow-up would be appreciated. ** Changed in: linux (Ubuntu) Status: Expired = Confirmed ** Tags added: kernel-fixed-upstream -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88 EFLAGS: 00010296 [153523.896276] RAX: RBX: RCX: 0082 [153523.896276] RDX: 12ea RSI: 81a6a3e9
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
[Expired for linux (Ubuntu) because there has been no activity for 60 days.] ** Changed in: linux (Ubuntu) Status: Incomplete = Expired -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Expired Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88 EFLAGS: 00010296 [153523.896276] RAX: RBX: RCX: 0082 [153523.896276] RDX: 12ea RSI: 81a6a3e9 RDI: [153523.896276] RBP: 88a97b18 R08: 81a74110 R09: 0005 [153523.896276] R10: ffe2 R11: 88a977fe R12: 0082 [153523.896276] R13: 81835280 R14: 12ea R15: 81a74110 [153523.896276] FS: 7ff43c02b840() GS:88000b80()
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
Are you able to reproduce the oops? If so, would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.14 kernel[0]. If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'. If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'. If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'. Once testing of the upstream kernel is complete, please mark this bug as Confirmed. Thanks in advance. [0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.14-trusty/ ** Changed in: linux (Ubuntu) Importance: Undecided = Medium ** Changed in: linux (Ubuntu) Status: Confirmed = Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Incomplete Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti:
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
** Changed in: linux (Ubuntu) Status: Incomplete = Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: Confirmed Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88 EFLAGS: 00010296 [153523.896276] RAX: RBX: RCX: 0082 [153523.896276] RDX: 12ea RSI: 81a6a3e9 RDI: [153523.896276] RBP: 88a97b18 R08: 81a74110 R09: 0005 [153523.896276] R10: ffe2 R11: 88a977fe R12: 0082 [153523.896276] R13: 81835280 R14: 12ea R15: 81a74110 [153523.896276] FS: 7ff43c02b840() GS:88000b80() knlGS: [153523.896276] CS: 0010 DS: ES: CR0:
[Kernel-packages] [Bug 1298972] Re: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0()
I'm attaching the dmesg extract as it reads more easily (same content as what's in the issue description). ** Attachment added: dmesg-extract.log https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1298972/+attachment/4048521/+files/dmesg-extract.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1298972 Title: stacktrace in ext4: /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() Status in “linux” package in Ubuntu: New Bug description: I was running an apt-get dist-upgrade when the FS suddenly remounted to RO. The 2 audit lines are from apt-dater running successively apt- get update and apt-get dist-upgrade: [153439.007924] type=1006 audit(1396010484.706:46): pid=16029 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=9 res=1 [153502.372488] type=1006 audit(1396010548.070:47): pid=16086 uid=0 old auid=4294967295 new auid=104 old ses=4294967295 new ses=10 res=1 [153523.874714] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 1, 6475 clusters in bitmap, 6473 in gd; block bitmap corrupt. [153523.876361] Aborting journal on device vda1-8. [153523.889719] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal [153523.890881] EXT4-fs (vda1): Remounting filesystem read-only [153523.891574] EXT4-fs (vda1): Remounting filesystem read-only [153523.892301] [ cut here ] [153523.892341] WARNING: CPU: 0 PID: 16129 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [153523.892342] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.892400] CPU: 0 PID: 16129 Comm: dpkg Not tainted 3.13.0-19-generic #40-Ubuntu [153523.892401] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.892406] 0009 88a97ad0 81711075 [153523.892408] 88a97b08 810662cd 88000aca3d00 [153523.892410] 88000a653000 81835280 12ea 88a97b18 [153523.892412] Call Trace: [153523.892444] [81711075] dump_stack+0x45/0x56 [153523.892461] [810662cd] warn_slowpath_common+0x7d/0xa0 [153523.892463] [810663aa] warn_slowpath_null+0x1a/0x20 [153523.892466] [81268112] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [153523.892483] [81240eca] ? ext4_dirty_inode+0x2a/0x60 [153523.892486] [812707d6] ext4_free_blocks+0x646/0xbf0 [153523.892488] [81261ec5] ext4_ext_rm_leaf+0x4a5/0x860 [153523.892490] [81260f87] ? __ext4_ext_check+0x197/0x330 [153523.892492] [812645e0] ? ext4_ext_remove_space+0xc0/0x7e0 [153523.892494] [8126483c] ext4_ext_remove_space+0x31c/0x7e0 [153523.892496] [81266bb0] ext4_ext_truncate+0xb0/0xe0 [153523.892498] [8123f339] ext4_truncate+0x379/0x3c0 [153523.892500] [8123fef1] ext4_evict_inode+0x491/0x4f0 [153523.892514] [811d3ac0] evict+0xb0/0x1b0 [153523.892516] [811d42d5] iput+0xf5/0x180 [153523.892525] [811c8cfe] do_unlinkat+0x18e/0x2b0 [153523.892537] [81020d25] ? syscall_trace_enter+0x145/0x250 [153523.892539] [811c9c56] SyS_unlink+0x16/0x20 [153523.892545] [81721c7f] tracesys+0xe1/0xe6 [153523.892547] ---[ end trace 9b25f1487620e754 ]--- [153523.892582] BUG: unable to handle kernel NULL pointer dereference at 0028 [153523.893579] IP: [81256fa1] __ext4_error_inode+0x31/0x160 [153523.894350] PGD a06c067 PUD 819d067 PMD 0 [153523.894933] Oops: [#1] SMP [153523.895397] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables psmouse serio_raw floppy [153523.896276] CPU: 0 PID: 16129 Comm: dpkg Tainted: GW 3.13.0-19-generic #40-Ubuntu [153523.896276] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [153523.896276] task: 880009455fc0 ti: 88a96000 task.ti: 88a96000 [153523.896276] RIP: 0010:[81256fa1] [81256fa1] __ext4_error_inode+0x31/0x160 [153523.896276] RSP: 0018:88a97a88 EFLAGS: 00010296 [153523.896276] RAX: RBX: RCX: 0082 [153523.896276] RDX: 12ea RSI: 81a6a3e9 RDI: [153523.896276] RBP: 88a97b18 R08: 81a74110 R09: 0005 [153523.896276] R10: ffe2 R11: 88a977fe R12: 0082 [153523.896276] R13: 81835280 R14: 12ea
