Not quite sure if it's the same bug, but maybe this adds some more data points to the set.
AMI ID is ami-405c6934, which is the eu-west-1 EBS-backed variant of the image in question. Just a plain instance boot, connecting four EBS volumes, bundling them together as /dev/md0, putting XFS on top and running a benchmark as described in http://www.mysqlperformanceblog.com/2009/08/06/ec2ebs- single-and-raid-volumes-io-bencmark/. When doing the same (on the same volumes) from a 32bit AMI on a .small instance the problem does not occur (at least not over a few days), whereas the 64bit AMI crashed within hours. Dec 5 22:00:55 ip-10-234-243-114 kernel: [ 6.642845] JBD: barrier-based sync failed on sda1-8 - disabling barriers Dec 5 22:01:02 ip-10-234-243-114 kernel: [ 13.890171] eth0: no IPv6 routers present Dec 5 22:04:14 ip-10-234-243-114 kernel: [ 205.786453] SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled Dec 5 22:04:14 ip-10-234-243-114 kernel: [ 205.789239] SGI XFS Quota Management subsystem Dec 5 22:04:14 ip-10-234-243-114 kernel: [ 205.790489] Filesystem "md0": Disabling barriers, trial barrier write failed Dec 5 22:04:14 ip-10-234-243-114 kernel: [ 205.807777] XFS mounting filesystem md0 Dec 5 22:04:15 ip-10-234-243-114 kernel: [ 206.368403] Ending clean XFS mount for filesystem: md0 Dec 5 22:07:36 ip-10-234-243-114 kernel: [ 407.339573] Filesystem "md0": Disabling barriers, trial barrier write failed Dec 5 22:07:36 ip-10-234-243-114 kernel: [ 407.340479] XFS mounting filesystem md0 Dec 5 22:07:36 ip-10-234-243-114 kernel: [ 407.593179] Ending clean XFS mount for filesystem: md0 Dec 5 22:17:01 ip-10-234-243-114 CRON[1222]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Dec 5 22:31:54 ip-10-234-243-114 kernel: [ 1865.812490] XFS mounting filesystem md0 Dec 5 22:31:54 ip-10-234-243-114 kernel: [ 1866.061941] Ending clean XFS mount for filesystem: md0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190047] INFO: task flush-9:0:1272 blocked for more than 120 seconds. Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190061] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190068] flush-9:0 D ffff880003e7d980 0 1272 2 0x00000000 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190072] ffff88014d79b640 0000000000000246 ffff880100000000 0000000000015980 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190077] ffff88014d79bfd8 0000000000015980 ffff88014d79bfd8 ffff8801d58316e0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190081] 0000000000015980 0000000000015980 ffff88014d79bfd8 0000000000015980 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190084] Call Trace: Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190095] [<ffffffff815a20f3>] io_schedule+0x73/0xc0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190099] [<ffffffff812a2f1c>] get_request_wait+0xcc/0x1a0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190104] [<ffffffff8107f080>] ? autoremove_wake_function+0x0/0x40 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190107] [<ffffffff812a3083>] __make_request+0x93/0x4b0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190111] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190113] [<ffffffff812a1c63>] generic_make_request+0x1b3/0x540 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190116] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190121] [<ffffffff81181862>] ? bvec_alloc_bs+0x62/0x110 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190125] [<ffffffff81142b07>] ? kmem_cache_alloc+0x77/0x120 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190127] [<ffffffff812a2072>] submit_bio+0x82/0x110 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190131] [<ffffffff811760a2>] ? __mark_inode_dirty+0x42/0x1d0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190173] [<ffffffffa00d2d37>] xfs_submit_ioend_bio+0x57/0x90 [xfs] Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190189] [<ffffffffa00d2e22>] xfs_submit_ioend+0xb2/0x110 [xfs] Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190205] [<ffffffffa00d3f98>] xfs_page_state_convert+0x348/0x6d0 [xfs] Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190222] [<ffffffffa00d44d5>] xfs_vm_writepage+0x95/0x180 [xfs] Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190225] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190230] [<ffffffff81109417>] __writepage+0x17/0x40 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190233] [<ffffffff8110a537>] write_cache_pages+0x1c7/0x3d0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190235] [<ffffffff81109400>] ? __writepage+0x0/0x40 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190239] [<ffffffff8110a764>] generic_writepages+0x24/0x30 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190254] [<ffffffffa00d335d>] xfs_vm_writepages+0x5d/0x80 [xfs] Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190257] [<ffffffff8110a791>] do_writepages+0x21/0x40 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190260] [<ffffffff81175356>] writeback_single_inode+0xe6/0x3f0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190264] [<ffffffff81175ab5>] writeback_sb_inodes+0x195/0x280 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190266] [<ffffffff811762d0>] writeback_inodes_wb+0xa0/0x1b0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190269] [<ffffffff8117662b>] wb_writeback+0x24b/0x2b0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190274] [<ffffffff810709b2>] ? del_timer_sync+0x22/0x30 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190277] [<ffffffff81176739>] wb_do_writeback+0xa9/0x190 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190279] [<ffffffff8106ffd0>] ? process_timeout+0x0/0x10 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190282] [<ffffffff81176873>] bdi_writeback_task+0x53/0x160 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190285] [<ffffffff8107ef47>] ? bit_waitqueue+0x17/0xd0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190289] [<ffffffff81119cb6>] bdi_start_fn+0x86/0x100 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190292] [<ffffffff81119c30>] ? bdi_start_fn+0x0/0x100 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190294] [<ffffffff8107eb26>] kthread+0x96/0xa0 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190298] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190301] [<ffffffff815a45dd>] ? retint_restore_args+0x5/0x6 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190304] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190307] INFO: task xfsbufd/md0:1323 blocked for more than 120 seconds. Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190313] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190319] xfsbufd/md0 D ffff880003e7d980 0 1323 2 0x00000000 Dec 5 23:04:48 ip-10-234-243-114 kernel: [ 3840.190323] ffff8801d761fb50 0000000000000246 ffffffff00000000 0000000000015980 ...skipping... Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200074] ffff8801d483bfd8 0000000000015980 ffff8801d483bfd8 ffff8801d5b10000 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200079] 0000000000015980 0000000000015980 ffff8801d483bfd8 0000000000015980 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200084] Call Trace: Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200096] [<ffffffff815a404e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200134] [<ffffffffa00c7147>] xlog_state_get_iclog_space+0xe7/0x2d0 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200140] [<ffffffff81056c10>] ? default_wake_function+0x0/0x20 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200167] [<ffffffffa00c7f74>] xlog_write+0x174/0x510 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200172] [<ffffffff812a1c63>] ? generic_make_request+0x1b3/0x540 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200176] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200191] [<ffffffffa00c8387>] xfs_log_write+0x77/0xa0 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200209] [<ffffffffa00d4af9>] xfs_trans_commit_iclog+0x169/0x300 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200225] [<ffffffffa00c89c4>] ? xlog_grant_log_space+0x3f4/0x5d0 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200241] [<ffffffffa00c8ca6>] ? xfs_log_reserve+0x106/0x170 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200256] [<ffffffffa00d546d>] _xfs_trans_commit+0x8d/0x2c0 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200273] [<ffffffffa00eb87e>] xfs_commit_dummy_trans+0x9e/0xf0 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200289] [<ffffffffa00ebf44>] xfs_sync_worker+0x74/0x80 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200305] [<ffffffffa00eb733>] xfssyncd+0x183/0x230 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200320] [<ffffffffa00eb5b0>] ? xfssyncd+0x0/0x230 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200324] [<ffffffff8107eb26>] kthread+0x96/0xa0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200327] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200331] [<ffffffff815a45dd>] ? retint_restore_args+0x5/0x6 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200334] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200351] INFO: task flush-9:0:1078 blocked for more than 120 seconds. Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200357] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200364] flush-9:0 D ffff880003e5f980 0 1078 2 0x00000000 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200367] ffff8801737ad640 0000000000000246 0000000000000000 0000000000015980 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200371] ffff8801737adfd8 0000000000015980 ffff8801737adfd8 ffff8801d75f96e0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200375] 0000000000015980 0000000000015980 ffff8801737adfd8 0000000000015980 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200378] Call Trace: Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200381] [<ffffffff815a20f3>] io_schedule+0x73/0xc0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200384] [<ffffffff812a2f1c>] get_request_wait+0xcc/0x1a0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200387] [<ffffffff8107f080>] ? autoremove_wake_function+0x0/0x40 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200389] [<ffffffff812a3083>] __make_request+0x93/0x4b0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200392] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200395] [<ffffffff812a1c63>] generic_make_request+0x1b3/0x540 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200398] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200402] [<ffffffff81181862>] ? bvec_alloc_bs+0x62/0x110 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200406] [<ffffffff81142b07>] ? kmem_cache_alloc+0x77/0x120 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200408] [<ffffffff812a2072>] submit_bio+0x82/0x110 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200412] [<ffffffff811760a2>] ? __mark_inode_dirty+0x42/0x1d0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200427] [<ffffffffa00ddd37>] xfs_submit_ioend_bio+0x57/0x90 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200441] [<ffffffffa00dde22>] xfs_submit_ioend+0xb2/0x110 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200456] [<ffffffffa00def98>] xfs_page_state_convert+0x348/0x6d0 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200471] [<ffffffffa00df4d5>] xfs_vm_writepage+0x95/0x180 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200474] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200478] [<ffffffff81109417>] __writepage+0x17/0x40 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200481] [<ffffffff8110a537>] write_cache_pages+0x1c7/0x3d0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200483] [<ffffffff81109400>] ? __writepage+0x0/0x40 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200487] [<ffffffff8110a764>] generic_writepages+0x24/0x30 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200501] [<ffffffffa00de35d>] xfs_vm_writepages+0x5d/0x80 [xfs] Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200504] [<ffffffff8110a791>] do_writepages+0x21/0x40 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200507] [<ffffffff81175356>] writeback_single_inode+0xe6/0x3f0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200510] [<ffffffff81175ab5>] writeback_sb_inodes+0x195/0x280 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200513] [<ffffffff811762d0>] writeback_inodes_wb+0xa0/0x1b0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200516] [<ffffffff8117662b>] wb_writeback+0x24b/0x2b0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200520] [<ffffffff810709b2>] ? del_timer_sync+0x22/0x30 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200523] [<ffffffff81176739>] wb_do_writeback+0xa9/0x190 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200526] [<ffffffff8106ffd0>] ? process_timeout+0x0/0x10 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200529] [<ffffffff81176873>] bdi_writeback_task+0x53/0x160 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200531] [<ffffffff8107ef47>] ? bit_waitqueue+0x17/0xd0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200535] [<ffffffff81119cb6>] bdi_start_fn+0x86/0x100 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200538] [<ffffffff81119c30>] ? bdi_start_fn+0x0/0x100 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200540] [<ffffffff8107eb26>] kthread+0x96/0xa0 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200542] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200546] [<ffffffff815a45dd>] ? retint_restore_args+0x5/0x6 Dec 6 00:36:00 ip-10-235-98-89 kernel: [ 3960.200548] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666211 Title: maverick on ec2 64bit ext4 deadlock -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
