Re: [gentoo-hardened] kernel upgrade problems: bad page state

[Brian Kroth]

Natanael Copa wrote:

On Sun, 2007-11-04 at 01:34 +0200, [EMAIL PROTECTED] wrote:

On 3 Nov 2007 at 19:02, Brian Kroth wrote:

PS - would you like me to take this off list?

no need unless you don't want to leak any more of your host names
in kernel logs ;-). actually, attachments this big you can send me
directly and don't bother the list with it, but the discussion may
be interesting for others as well.

This discussion is interesting, yes

-nc

Nov  5 09:58:42 tux-mc PAX: 19300 14a1edf0 1200e000-15000000 0001200e


Here's what I've gotten on startup since then:

Nov  5 11:14:31 tux-mc PAX: 6190 11e13d40 11de2000-11e58000 00011de2
Nov  5 11:14:32 tux-mc PAX: 6274 4abafeb0 4abaf000-4abb1000 00000002
Nov  5 11:14:34 tux-mc IPMI Watchdog: driver initialized
Nov  5 11:14:56 tux-mc PAX: 5200 16d4796c 16d38000-16d64000 00016d38

Nov  5 11:15:20 tux-mc printing eip:
Nov  5 11:15:20 tux-mc c06ee937
Nov  5 11:15:20 tux-mc *pde = 00000000
Nov  5 11:15:20 tux-mc Oops: 0002 [#1]
Nov  5 11:15:20 tux-mc SMP DEBUG_PAGEALLOC

Nov  5 11:15:20 tux-mc CPU:    1
Nov  5 11:15:20 tux-mc EIP:    0060:[<c06ee937>]    Not tainted VLI
Nov  5 11:15:20 tux-mc EFLAGS: 00255692   (2.6.23-hardened-r1 #3)
Nov  5 11:15:20 tux-mc EIP is at __down+0xcd/0xdb

Nov  5 11:15:20 tux-mc ds: 0068   es: 0068   fs: 00d8  gs: 0000  ss: 0068

Nov  5 11:15:20 tux-mc Call Trace:
Nov  5 11:15:20 tux-mc [<c0403e94>] show_trace_log_lvl+0x1a/0x30
Nov  5 11:15:20 tux-mc [<c0403f4b>] show_stack_log_lvl+0x8d/0xaa
Nov  5 11:15:20 tux-mc [<c04041ca>] show_registers+0x212/0x378
Nov  5 11:15:20 tux-mc [<c04044d6>] die+0x121/0x208
Nov  5 11:15:20 tux-mc [<c06f06f5>] do_page_fault+0x2dd/0x80a
Nov  5 11:15:20 tux-mc [<c06eedb5>] error_code+0x75/0x80
Nov  5 11:15:20 tux-mc [<f7f51ef8>] 0xf7f51ef8
Nov  5 11:15:20 tux-mc =======================

Nov  5 11:15:39 tux-mc PAX: 7110 500cf1d0 500cf000-500d0000 00000015
Nov  5 11:16:21 tux-mc PAX: 7307 13565f30 134ae000-13b89000 000134ae
Nov  5 11:20:01 tux-mc PAX: 8336 51626edc 51626000-51628000 00000004
Nov  5 11:20:10 tux-mc PAX: 8719 55ee5d88 55ee4000-55ee7000 0000012d
Nov  5 11:25:57 tux-mc PAX: 11103 500cf1d0 500cf000-500d0000 00000015
Nov  5 11:30:10 tux-mc PAX: 12755 53b06e38 53300000-53b65000 00053300
Nov  5 11:30:10 tux-mc PAX: 12753 53b06990 53300000-53b65000 00053300
Nov  5 11:30:10 tux-mc Bad page state in process 'cactid'

Nov  5 11:30:10 tux-mc Trying to fix it up, but a reboot is needed
Nov  5 11:30:10 tux-mc Backtrace:
Nov  5 11:30:10 tux-mc [<c0403e94>] show_trace_log_lvl+0x1a/0x30
Nov  5 11:30:10 tux-mc [<c0403ebc>] show_trace+0x12/0x14
Nov  5 11:30:10 tux-mc [<c0403fb6>] dump_stack+0x16/0x18
Nov  5 11:30:10 tux-mc [<c044bbd5>] bad_page+0x66/0x97
Nov  5 11:30:10 tux-mc [<c044c5c7>] free_hot_cold_page+0x71/0x176
Nov  5 11:30:10 tux-mc [<c044c6d6>] free_hot_page+0xa/0xc
Nov  5 11:30:10 tux-mc [<c044f18c>] __page_cache_release+0x84/0x90
Nov  5 11:30:10 tux-mc [<c044f1f8>] put_page+0x27/0x36
Nov  5 11:30:10 tux-mc [<c0455da8>] do_wp_page+0x235/0x48f
Nov  5 11:30:10 tux-mc [<c0457233>] handle_mm_fault+0x2bc/0x2ef
Nov  5 11:30:10 tux-mc [<c06f0764>] do_page_fault+0x34c/0x80a
Nov  5 11:30:10 tux-mc [<c06eedb5>] error_code+0x75/0x80
Nov  5 11:30:10 tux-mc =======================
Nov  5 11:35:43 tux-mc PAX: 5743 5da97f48 5da5d000-5da9f000 000bffbe
Nov  5 11:40:37 tux-mc PAX: 5744 134ae718 134ae000-13baa000 000134ae
Nov  5 11:40:37 tux-mc PAX: 16212 134ae300 134ae000-13baa000 000134ae
Nov  5 11:40:37 tux-mc Bad page state in process 'nagios'

Nov  5 11:40:37 tux-mc Trying to fix it up, but a reboot is needed
Nov  5 11:40:37 tux-mc Backtrace:
Nov  5 11:40:37 tux-mc [<c0403e94>] show_trace_log_lvl+0x1a/0x30
Nov  5 11:40:37 tux-mc [<c0403ebc>] show_trace+0x12/0x14
Nov  5 11:40:37 tux-mc [<c0403fb6>] dump_stack+0x16/0x18
Nov  5 11:40:37 tux-mc [<c044bbd5>] bad_page+0x66/0x97
Nov  5 11:40:37 tux-mc [<c044c5c7>] free_hot_cold_page+0x71/0x176
Nov  5 11:40:37 tux-mc [<c044c6d6>] free_hot_page+0xa/0xc
Nov  5 11:40:37 tux-mc [<c044f18c>] __page_cache_release+0x84/0x90
Nov  5 11:40:37 tux-mc [<c044f1f8>] put_page+0x27/0x36
Nov  5 11:40:37 tux-mc [<c0455da8>] do_wp_page+0x235/0x48f
Nov  5 11:40:37 tux-mc [<c0457233>] handle_mm_fault+0x2bc/0x2ef
Nov  5 11:40:37 tux-mc [<c06f0764>] do_page_fault+0x34c/0x80a
Nov  5 11:40:37 tux-mc [<c06eedb5>] error_code+0x75/0x80
Nov  5 11:40:37 tux-mc =======================

Getting more interesting. It just rebooted itself, which it hasn't done yet. Here's the two messages in kern.log before and after it happened. Nothing unexpected in any of the other log files either. Nov 5 11:14:02 tux-mc Linux version 2.6.23-hardened-r1 ([EMAIL PROTECTED]) (gcc version 3.4.6 (Gentoo Hardened 3.4.6-r2, ssp-3.4.6-1.0, pie-8.7.10)) #3 SMP Sun Nov 4 13:01:03 CST 2007 Nov 5 11:15:20 tux-mc BUG: unable to handle kernel NULL pointer dereference at virtual address 00000c4a Nov 5 11:15:20 tux-mc Modules linked in: ipmi_watchdog bonding ipmi_si ipmi_devintf ipmi_msghandler e1000 Nov 5 11:15:20 tux-mc eax: 00000c4a ebx: c0419295 ecx: f7add000 edx: f1571500 Nov 5 11:15:20 tux-mc esi: 57c7b46d edi: 00000c4a ebp: f7f51ed8 esp: f7f51ea8 Nov 5 11:15:20 tux-mc Process perl (pid: 6953, ti=f7f50000 task=f5fba000 task.ti=d299c000) Nov 5 11:15:20 tux-mc Stack: 00000c4a 00000000 00000f85 f7add030 c180bd00 f7f51ef8 c0415117 00000c4a Nov 5 11:15:20 tux-mc 00000000 c180bd84 00000000 50a3f400 00000408 f7f51ef8 c053bf61 00000408 Nov 5 11:15:20 tux-mc 00000000 c180bd20 c180bd20 c180bd00 f7f51f20 c04152bd 000220f2 00000000 Nov 5 11:15:20 tux-mc Code: 89 50 04 89 02 89 f0 ba 03 00 00 00 c7 45 e0 00 01 10 00 c7 45 e4 00 02 20 00 e8 4b a9 d2 ff 89 f0 89 fa e8 af 02 00 00 8b 45 d0 <c7> 00 00 00 00 00 83 c4 24 5b 5e 5f 5d c3 55 b9 05 00 00 00 89 Nov 5 11:15:20 tux-mc EIP: [<c06ee937>] __down+0xcd/0xdb SS:ESP 0068:f7f51ea8 Nov 5 11:30:10 tux-mc page:c104e3a0 flags:0x40000001 mapping:00000000 mapcount:0 count:0 Nov 5 11:40:37 tux-mc page:c1084060 flags:0x40000001 mapping:00000000 mapcount:0 count:0

smime.p7s
Description: S/MIME Cryptographic Signature