Source: linux-2.6 Version: 2.6.32-5 Severity: normal
Hello, Today one of my servers stopped responding to some webs, ssh'ing into it was impossible. The ping worked and the ssh connection was starting but the shell didn't showed up after waiting a long time. Finally a hard-reset was needed in order to bring it back After the reboot I found this on kern.log # tail /var/log/kern.log Feb 1 09:41:00 server-i7_920 kernel: [17383037.287331] EXT4-fs (dm-29): mounted filesystem with ordered data mode Feb 5 09:38:15 server-i7_920 kernel: [17727613.769052] NOHZ: local_softirq_pending 100 Feb 7 05:56:07 server-i7_920 kernel: [17886689.887577] e1000e: eth0 NIC Link is Down Feb 7 05:59:05 server-i7_920 kernel: [17886867.166464] e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None Feb 7 05:59:41 server-i7_920 kernel: [17886903.722727] e1000e: eth0 NIC Link is Down Feb 7 06:00:00 server-i7_920 kernel: [17886922.309159] e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876326] BUG: soft lockup - CPU#7 stuck for 17163091979s! [init:9709] Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876371] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs ext4 jbd2 crc16 ext2 ipt_LOG sg xt_limit xt_tcpudp xt_state iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables x_tables dummy loop snd_pcm snd_timer snd soundcore snd_page_alloc ioatdma i2c_i801 i2c_core pcspkr dca psmouse evdev button serio_raw processor ext3 jbd mbcache dm_mod sd_mod crc_t10dif ahci libata uhci_hcd ehci_hcd scsi_mod usbcore nls_base e1000e thermal thermal_sys [last unloaded: scsi_wait_scan] Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876405] CPU 7: Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876406] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs ext4 jbd2 crc16 ext2 ipt_LOG sg xt_limit xt_tcpudp xt_state iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables x_tables dummy loop snd_pcm snd_timer snd soundcore snd_page_alloc ioatdma i2c_i801 i2c_core pcspkr dca psmouse evdev button serio_raw processor ext3 jbd mbcache dm_mod sd_mod crc_t10dif ahci libata uhci_hcd ehci_hcd scsi_mod usbcore nls_base e1000e thermal thermal_sys [last unloaded: scsi_wait_scan] Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876433] Pid: 9709, comm: init Not tainted 2.6.32-5-vserver-amd64 #1 X8STi Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876435] RIP: 0023:[<00000000f76c0430>] [<00000000f76c0430>] 0xf76c0430 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876440] RSP: 002b:00000000ffa6673c EFLAGS: 00000296 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876442] RAX: fffffffffffffff6 RBX: 0000000000000000 RCX: 00000000ffa6673c Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876443] RDX: 00000000f76c0430 RSI: 000000000000000f RDI: 0000000000000000 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876445] RBP: ffffffff8101166e R08: 0000000000000000 R09: 0000000000000000 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876447] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876448] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876451] FS: 0000000000000000(0000) GS:ffff880016bc0000(0063) knlGS:00000000f74a96c0 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876453] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876454] CR2: 00007fa1566220a0 CR3: 0000000514e5d000 CR4: 00000000000006e0 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876456] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876458] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876460] Call Trace: Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130843] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130874] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130919] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130923] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130927] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130931] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130934] Call Trace: Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130942] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130947] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130950] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131009] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131013] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131016] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131021] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131025] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131028] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832664] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832694] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832797] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832802] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832806] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832809] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832813] Call Trace: Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832820] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832825] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832829] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832833] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832837] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832840] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832845] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832849] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832852] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537240] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537271] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537316] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537320] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537324] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537328] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537331] Call Trace: Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537339] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537344] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537347] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537352] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537356] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537359] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537364] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537367] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537370] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236311] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236342] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236387] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236392] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236395] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236399] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236403] Call Trace: Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236410] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236415] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236418] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236423] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236427] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236430] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236435] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236439] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236442] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938131] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938162] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938207] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938211] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938215] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938219] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938223] Call Trace: Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938230] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938235] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938238] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938243] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938247] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938250] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938255] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938258] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938261] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Feb 8 18:30:04 server-i7_920 kernel: [ 3601.643990] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644021] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644066] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644070] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644074] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644078] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644081] Call Trace: Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644089] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644094] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644097] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644101] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644105] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644108] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644113] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644117] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644120] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346325] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346363] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346408] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346413] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346417] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346420] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346424] Call Trace: Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346431] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346437] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346440] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346444] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346449] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346452] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346457] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346460] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346463] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043601] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043632] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043677] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043682] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043685] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043689] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043693] Call Trace: Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043700] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043705] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043708] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043713] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043717] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043720] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043725] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043729] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043732] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745606] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745637] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745682] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745686] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745690] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745694] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745698] Call Trace: Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745705] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745710] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745713] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745718] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745722] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745725] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745730] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745734] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745737] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447611] INFO: task cron:8748 blocked for more than 120 seconds. Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447642] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447687] cron D 0000000000000000 0 8748 25441 0x00020000 Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447692] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000 Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447696] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8 Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447699] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8 Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447703] Call Trace: Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447710] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447715] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447718] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447723] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9 Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447727] [<ffffffff8103a4af>] ? activate_task+0x22/0x28 Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447730] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447735] [<ffffffff8101657d>] ? read_tsc+0xa/0x20 Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447738] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22 Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447741] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c Notice the weird thing between 17:28:57 and 18:20:04. The [timestamps] seems to go backwards (!!) How this can be even possible?... Also many of the modules listed on "Modules linked in" (btrfs,ufs,qnx4..) were *NOT* expected to be in use. This is some useful information about this server (got after the reboot): # lsmod Module Size Used by xt_limit 1782 1 xt_tcpudp 2319 9 xt_state 1303 2 iptable_mangle 2817 0 iptable_nat 4299 1 nf_nat 13388 1 iptable_nat nf_conntrack_ipv4 9833 5 iptable_nat,nf_nat nf_conntrack 46551 4 xt_state,iptable_nat,nf_nat,nf_conntrack_ipv4 nf_defrag_ipv4 1139 1 nf_conntrack_ipv4 iptable_filter 2258 1 ip_tables 13915 3 iptable_mangle,iptable_nat,iptable_filter x_tables 12845 5 xt_limit,xt_tcpudp,xt_state,iptable_nat,ip_tables ext4 291462 5 jbd2 67015 1 ext4 crc16 1319 1 ext4 dummy 1584 0 loop 11975 0 snd_pcm 60503 0 i2c_i801 7830 0 snd_timer 15582 1 snd_pcm i2c_core 15819 1 i2c_i801 snd 46526 2 snd_pcm,snd_timer soundcore 4598 1 snd snd_page_alloc 6249 1 snd_pcm pcspkr 1699 0 ioatdma 34892 24 dca 3761 1 ioatdma psmouse 49937 0 evdev 7352 2 serio_raw 3752 0 processor 29951 8 button 4650 0 ext3 109110 25 jbd 37101 1 ext3 mbcache 5050 2 ext4,ext3 dm_mod 54320 91 sd_mod 29921 3 crc_t10dif 1276 1 sd_mod uhci_hcd 18537 0 ahci 32534 2 libata 133776 1 ahci ehci_hcd 32097 0 scsi_mod 126533 2 sd_mod,libata usbcore 122514 3 uhci_hcd,ehci_hcd nls_base 6377 1 usbcore thermal 11674 0 e1000e 110079 0 thermal_sys 11942 2 processor,thermal # parted /dev/sda print Model: ATA ST32000641AS (scsi) Disk /dev/sda: 2000GB Sector size (logical/physical): 512B/512B Partition Table: gpt Number Start End Size File system Name Flags 1 17.4kB 512MB 512MB ext3 boot 2 512MB 2000GB 2000GB lvm # pvs PV VG Fmt Attr PSize PFree /dev/sda2 vg lvm2 a- 1.82t 1.46t # uname -r 2.6.32-5-vserver-amd64 Thanks! -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Carlos Alberto Lopez Perez http://neutrino.es Igalia - Free Software Engineering http://www.igalia.com ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
signature.asc
Description: OpenPGP digital signature
