This problem just happened again. Here is an extract from /var/log/syslog on the machine affected. As before, the other virtual machine on this host was ok.
Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] BUG: soft lockup - CPU#1 stuck for 61s! [fcheck:21720] Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] Modules linked in: ipv6 evdev psmouse serio_raw button ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif sg ata_generic uhci_ hcd ata_piix e1000 usbcore libata scsi_mod dock thermal processor fan Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] CPU 1: Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] Modules linked in: ipv6 evdev psmouse serio_raw button ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif sg ata_generic uhci_ hcd ata_piix e1000 usbcore libata scsi_mod dock thermal processor fan Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] Pid: 21720, comm: fcheck Not tainted 2.6.27-11-server #1 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] RIP: 0010:[<ffffffff802abeb7>] [<ffffffff802abeb7>] find_get_pages+0x77/0x110 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] RSP: 0000:ffff880003d37948 EFLAGS: 00000293 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] RAX: ffff880006aaa5e8 RBX: ffff880003d37988 RCX: ffff880006aaa5e8 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffe20000126900 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] RBP: ffff880003d378f8 R08: 0000000000000002 R09: 0000000000000001 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] R10: 0000000000000002 R11: ffff880003d37a78 R12: ffffffff802b6d84 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] R13: ffff880003d37988 R14: 0000000000000001 R15: 0000000000000800 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] FS: 00007f1991ff36e0(0000) GS:ffff88000f495180(0000) knlGS:0000000000000000 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] CR2: 0000000000dfb190 CR3: 000000000f15e000 CR4: 00000000000006e0 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] Call Trace: Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802abe83>] ? find_get_pages+0x43/0x110 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802b6a04>] ? pagevec_lookup+0x24/0x30 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802b825b>] ? __invalidate_mapping_pages+0x8b/0x1a0 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802b4874>] ? get_dirty_limits+0x14/0x2b0 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff803024ae>] ? generic_forget_inode+0x4e/0x190 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802b8380>] ? invalidate_mapping_pages+0x10/0x20 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff803022ed>] ? prune_icache+0x27d/0x290 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff8030233f>] ? shrink_icache_memory+0x3f/0x50 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802b8d45>] ? shrink_slab+0x135/0x190 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802ba3af>] ? do_try_to_free_pages+0x17f/0x2e0 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802ba607>] ? try_to_free_pages+0x67/0x70 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802b9120>] ? isolate_pages_global+0x0/0x50 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802b2931>] ? __alloc_pages_internal+0x241/0x510 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802d6679>] ? alloc_page_vma+0x79/0xf0 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802c2557>] ? do_wp_page+0x187/0x640 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802c3160>] ? handle_mm_fault+0x3b0/0x470 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff80505e4f>] ? do_page_fault+0x34f/0x750 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff80246463>] ? task_new_fair+0xb3/0x100 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff80245e5d>] ? resched_task+0x2d/0x90 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802fc3d1>] ? locks_remove_posix+0x11/0xc0 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff80240cc8>] ? wake_up_new_task+0xc8/0x140 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802f0dc2>] ? pipe_ioctl+0x12/0xb0 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802f86b6>] ? vfs_ioctl+0x36/0xb0 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff805030be>] ? _spin_lock+0xe/0x20 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff802f8135>] ? set_close_on_exec+0x75/0xb0 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] [<ffffffff805034fa>] ? error_exit+0x0/0x70 Mar 3 22:32:25 gla1-mailman1 kernel: [740722.950322] Mar 3 22:33:13 gla1-mailman1 postfix/anvil[21708]: statistics: max connection rate 2/60s for (smtp:10.2.8.16) at Mar 3 22:29:51 Mar 3 22:33:13 gla1-mailman1 postfix/anvil[21708]: statistics: max connection count 1 for (smtp:10.2.8.16) at Mar 3 22:29:44 Mar 3 22:33:13 gla1-mailman1 postfix/anvil[21708]: statistics: max cache size 1 at Mar 3 22:29:44 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450326] BUG: soft lockup - CPU#1 stuck for 61s! [fcheck:21720] Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] Modules linked in: ipv6 evdev psmouse serio_raw button ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif sg ata_generic uhci_hcd ata_piix e1000 usbcore libata scsi_mod dock thermal processor fan Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] CPU 1: Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] Modules linked in: ipv6 evdev psmouse serio_raw button ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif sg ata_generic uhci_hcd ata_piix e1000 usbcore libata scsi_mod dock thermal processor fan Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] Pid: 21720, comm: fcheck Not tainted 2.6.27-11-server #1 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] RIP: 0010:[<ffffffff802abeac>] [<ffffffff802abeac>] find_get_pages+0x6c/0x110 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] RSP: 0000:ffff880003d37948 EFLAGS: 00000246 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] RAX: ffff880006aaa5e8 RBX: ffff880003d37988 RCX: ffff880006aaa5e8 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffe20000126900 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] RBP: ffff880003d378f8 R08: 0000000000000002 R09: 0000000000000001 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] R10: 0000000000000002 R11: ffff880003d37a78 R12: ffffffff802b6d84 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] R13: ffff880003d37988 R14: 0000000000000001 R15: 0000000000000800 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] FS: 00007f1991ff36e0(0000) GS:ffff88000f495180(0000) knlGS:0000000000000000 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] CR2: 0000000000dfb190 CR3: 000000000f15e000 CR4: 00000000000006e0 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] Call Trace: Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802abe83>] ? find_get_pages+0x43/0x110 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802b6a04>] ? pagevec_lookup+0x24/0x30 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802b825b>] ? __invalidate_mapping_pages+0x8b/0x1a0 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802b4874>] ? get_dirty_limits+0x14/0x2b0 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff803024ae>] ? generic_forget_inode+0x4e/0x190 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802b8380>] ? invalidate_mapping_pages+0x10/0x20 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff803022ed>] ? prune_icache+0x27d/0x290 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff8030233f>] ? shrink_icache_memory+0x3f/0x50 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802b8d45>] ? shrink_slab+0x135/0x190 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802ba3af>] ? do_try_to_free_pages+0x17f/0x2e0 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802ba607>] ? try_to_free_pages+0x67/0x70 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802b9120>] ? isolate_pages_global+0x0/0x50 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802b2931>] ? __alloc_pages_internal+0x241/0x510 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802d6679>] ? alloc_page_vma+0x79/0xf0 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802c2557>] ? do_wp_page+0x187/0x640 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802c3160>] ? handle_mm_fault+0x3b0/0x470 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff80505e4f>] ? do_page_fault+0x34f/0x750 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff80246463>] ? task_new_fair+0xb3/0x100 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff80245e5d>] ? resched_task+0x2d/0x90 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802fc3d1>] ? locks_remove_posix+0x11/0xc0 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff80240cc8>] ? wake_up_new_task+0xc8/0x140 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802f0dc2>] ? pipe_ioctl+0x12/0xb0 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802f86b6>] ? vfs_ioctl+0x36/0xb0 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff805030be>] ? _spin_lock+0xe/0x20 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff802f8135>] ? set_close_on_exec+0x75/0xb0 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] [<ffffffff805034fa>] ? error_exit+0x0/0x70 Mar 3 22:33:30 gla1-mailman1 kernel: [740788.450367] The text above is repeated until 23:54:17, where I rebooted the virtual machine. A reboot makes the problem go away for a while. I have attached a screenshot too. I'm really keen to get this fixed. I'm happy to go through all sorts of debugging, this is a really important issue for my company. If you need any more details, please just asked and I'll do my best to gather the information needed. ** Attachment added: "soft_lockup.png" http://launchpadlibrarian.net/23398658/soft_lockup.png -- Virtual machine soft lockup - CPU gets stuck for XX seconds https://bugs.launchpad.net/bugs/333201 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
