lf_advlockasync panic [Was: pcpu.h kernel crash with 7.2]
On Fri, Sep 18, 2009 at 10:43:03AM +1000, Aristedes Maniatis wrote: This is a FreeBSD 7.2 machine in production. I'm not an expert at debugging kernel problems, but I've still got the vmcore if there is anything else I should run on it to extract more information. It is believed that the problem is fixed in RELENG_7 branch. If the issue is reproducable, it would worth the efforts to upgrade at least the kernel to latest RELENG_7 and retest. Thanks Ari Maniatis # uname -a FreeBSD dash.internal 7.2-RELEASE-p2 FreeBSD 7.2-RELEASE-p2 #0: Wed Jun 24 00:14:35 UTC 2009 r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 #kgdb /boot/kernel/kernel /var/crash/vmcore.3 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as amd64-marcel-freebsd... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 4; apic id = 04 fault virtual address = 0x11 fault code= supervisor read data, page not present instruction pointer = 0x8:0x804fbec9 stack pointer = 0x10:0x7b6a2830 frame pointer = 0x10:0x1 code segment = base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 69329 (httpd) trap number = 12 panic: page fault cpuid = 4 Uptime: 34d3h21m46s Physical memory: 24561 MB Dumping 5146 MB: 5131 5115 5099 5083 5067 5051 5035 5019 5003 4987 4971 4955 4939 4923 4907 4891 4875 4859 4843 4827 4811 4795 4779 4763 4747 4731 4715 4699 4683 4667 4651 4635 4619 4603 4587 4571 4555 4539 4523 4507 4491 4475 4459 4443 4427 4411 4395 4379 4363 4347 4331 4315 4299 4283 4267 4251 4235 4219 4203 4187 4171 4155 4139 4123 4107 4091 4075 4059 4043 4027 4011 3995 3979 3963 3947 3931 3915 3899 3883 3867 3851 3835 3819 3803 3787 3771 3755 3739 3723 3707 3691 3675 3659 3643 3627 3611 3595 3579 3563 3547 3531 3515 3499 3483 3467 3451 3435 3419 3403 3387 3371 3355 3339 3323 3307 3291 3275 3259 3243 3227 3211 3195 3179 3163 3147 3131 3115 3099 3083 3067 3051 3035 3019 3003 2987 2971 2955 2939 2923 2907 2891 2875 2859 2843 2827 2811 2795 2779 2763 2747 2731 2715 2699 2683 2667 2651 2635 2619 2603 2587 2571 2555 2539 2523 2507 2491 2475 2459 2443 2427 2411 2395 2379 2363 2347 2331 2315 2299 2283 2267 2251 2235 2219 2203 2187 2171 2155 2139 2123 2107 2091 2075 2059 2043 202 7 2011 1995 1979 1963 1947 1931 1915 1899 1883 1867 1851 1835 1819 1803 1787 1771 1755 1739 1723 1707 1691 1675 1659 1643 1627 1611 1595 1579 1563 1547 1531 1515 1499 1483 1467 1451 1435 1419 1403 1387 1371 1355 1339 1323 1307 1291 1275 1259 1243 1227 1211 1195 1179 1163 1147 1131 1115 1099 1083 1067 1051 1035 1019 1003 987 971 955 939 923 907 891 875 859 843 827 811 795 779 763 747 731 715 699 683 667 651 635 619 603 587 571 555 539 523 507 491 475 459 443 427 411 395 379 363 347 331 315 299 283 267 251 235 219 203 187 171 155 139 123 107 91 75 59 43 27 11 Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /bootdir/boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /bootdir/boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/accf_http.ko...Reading symbols from /bootdir/boot/kernel/accf_http.ko.symbols...done. done. Loaded symbols for /boot/kernel/accf_http.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /bootdir/boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko Reading symbols from /boot/kernel/smbus.ko...Reading symbols from /bootdir/boot/kernel/smbus.ko.symbols...done. done. Loaded symbols for /boot/kernel/smbus.ko Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /bootdir/boot/kernel/pflog.ko.symbols...done. done. Loaded symbols for /boot/kernel/pflog.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /bootdir/boot/kernel/pf.ko.symbols...done. done. Loaded symbols for /boot/kernel/pf.ko #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:195 #1 0x0004 in ?? () #2 0x8050df79 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #3 0x8050e382 in panic (fmt=0x104 Address 0x104 out of bounds) at /usr/src/sys/kern/kern_shutdown.c:574 #4 0x807d2253 in trap_fatal
pcpu.h kernel crash with 7.2
This is a FreeBSD 7.2 machine in production. I'm not an expert at debugging kernel problems, but I've still got the vmcore if there is anything else I should run on it to extract more information. Thanks Ari Maniatis # uname -a FreeBSD dash.internal 7.2-RELEASE-p2 FreeBSD 7.2-RELEASE-p2 #0: Wed Jun 24 00:14:35 UTC 2009 r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 #kgdb /boot/kernel/kernel /var/crash/vmcore.3 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as amd64-marcel-freebsd... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 4; apic id = 04 fault virtual address = 0x11 fault code = supervisor read data, page not present instruction pointer = 0x8:0x804fbec9 stack pointer = 0x10:0x7b6a2830 frame pointer = 0x10:0x1 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 69329 (httpd) trap number = 12 panic: page fault cpuid = 4 Uptime: 34d3h21m46s Physical memory: 24561 MB Dumping 5146 MB: 5131 5115 5099 5083 5067 5051 5035 5019 5003 4987 4971 4955 4939 4923 4907 4891 4875 4859 4843 4827 4811 4795 4779 4763 4747 4731 4715 4699 4683 4667 4651 4635 4619 4603 4587 4571 4555 4539 4523 4507 4491 4475 4459 4443 4427 4411 4395 4379 4363 4347 4331 4315 4299 4283 4267 4251 4235 4219 4203 4187 4171 4155 4139 4123 4107 4091 4075 4059 4043 4027 4011 3995 3979 3963 3947 3931 3915 3899 3883 3867 3851 3835 3819 3803 3787 3771 3755 3739 3723 3707 3691 3675 3659 3643 3627 3611 3595 3579 3563 3547 3531 3515 3499 3483 3467 3451 3435 3419 3403 3387 3371 3355 3339 3323 3307 3291 3275 3259 3243 3227 3211 3195 3179 3163 3147 3131 3115 3099 3083 3067 3051 3035 3019 3003 2987 2971 2955 2939 2923 2907 2891 2875 2859 2843 2827 2811 2795 2779 2763 2747 2731 2715 2699 2683 2667 2651 2635 2619 2603 2587 2571 2555 2539 2523 2507 2491 2475 2459 2443 2427 2411 2395 2379 2363 2347 2331 2315 2299 2283 2267 2251 2235 2219 2203 2187 2171 2155 2139 2123 2107 2091 2075 2059 2043 202 7 2011 1995 1979 1963 1947 1931 1915 1899 1883 1867 1851 1835 1819 1803 1787 1771 1755 1739 1723 1707 1691 1675 1659 1643 1627 1611 1595 1579 1563 1547 1531 1515 1499 1483 1467 1451 1435 1419 1403 1387 1371 1355 1339 1323 1307 1291 1275 1259 1243 1227 1211 1195 1179 1163 1147 1131 1115 1099 1083 1067 1051 1035 1019 1003 987 971 955 939 923 907 891 875 859 843 827 811 795 779 763 747 731 715 699 683 667 651 635 619 603 587 571 555 539 523 507 491 475 459 443 427 411 395 379 363 347 331 315 299 283 267 251 235 219 203 187 171 155 139 123 107 91 75 59 43 27 11 Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /bootdir/boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /bootdir/boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/accf_http.ko...Reading symbols from /bootdir/boot/kernel/accf_http.ko.symbols...done. done. Loaded symbols for /boot/kernel/accf_http.ko Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from /bootdir/boot/kernel/ipmi.ko.symbols...done. done. Loaded symbols for /boot/kernel/ipmi.ko Reading symbols from /boot/kernel/smbus.ko...Reading symbols from /bootdir/boot/kernel/smbus.ko.symbols...done. done. Loaded symbols for /boot/kernel/smbus.ko Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /bootdir/boot/kernel/pflog.ko.symbols...done. done. Loaded symbols for /boot/kernel/pflog.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /bootdir/boot/kernel/pf.ko.symbols...done. done. Loaded symbols for /boot/kernel/pf.ko #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:195 #1 0x0004 in ?? () #2 0x8050df79 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #3 0x8050e382 in panic (fmt=0x104 Address 0x104 out of bounds) at /usr/src/sys/kern/kern_shutdown.c:574 #4 0x807d2253 in trap_fatal (frame=0xff0315455370, eva=Variable eva is not available. ) at /usr/src/sys/amd64/amd64/trap.c:757 #5 0x807d2625 in trap_pfault (frame=0x7b6a2780, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:673 #6 0x807d2f64 in trap (frame=0x7b6a2780) at /usr/src/sys/amd64/amd64/trap.c:444 #7 0x807b70ce in calltrap () at
