Re: 11.0-CURRENT r290039 privileged instruction fault while in kernel mode
On Tue, Oct 27, 2015 at 04:09:28PM -0700, Don Lewis wrote: > I just got this crash while running poudriere on a freshly upgraded > 11.0-CURRENT machine. The instruction pointer value looks pretty > strange. > > > FreeBSD zipper.catspoiler.org 11.0-CURRENT FreeBSD 11.0-CURRENT #30 r290039: > Tue Oct 27 00:08:00 PDT 2015 > d...@zipper.catspoiler.org:/usr/obj/usr/src/sys/GENERIC amd64 > > panic: > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > > Fatal trap 1: privileged instruction fault while in kernel mode > cpuid = 4; apic id = 14 > instruction pointer = 0x20:0x8240fef5 What is the instruction at the reported address ? > stack pointer = 0x28:0xfe0859c636c0 > frame pointer = 0x28:0xfe0859c636e0 > code segment = base 0x0, limit 0xf, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 0 (zio_free_issue_2_9) > > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/zfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from > /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from > /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. > done. > Loaded symbols for /boot/kernel/geom_mirror.ko > Reading symbols from /boot/modules/vboxdrv.ko...done. > Loaded symbols for /boot/modules/vboxdrv.ko > Reading symbols from /boot/modules/vboxnetflt.ko...done. > Loaded symbols for /boot/modules/vboxnetflt.ko > Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from > /usr/lib/debug//boot/kernel/netgraph.ko.debug...done. > done. > Loaded symbols for /boot/kernel/netgraph.ko > Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from > /usr/lib/debug//boot/kernel/ng_ether.ko.debug...done. > done. > Loaded symbols for /boot/kernel/ng_ether.ko > Reading symbols from /boot/modules/vboxnetadp.ko...done. > Loaded symbols for /boot/modules/vboxnetadp.ko > Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/nullfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/nullfs.ko > Reading symbols from /boot/kernel/linux.ko...Reading symbols from > /usr/lib/debug//boot/kernel/linux.ko.debug...done. > done. > Loaded symbols for /boot/kernel/linux.ko > Reading symbols from /boot/kernel/linux_common.ko...Reading symbols from > /usr/lib/debug//boot/kernel/linux_common.ko.debug...done. > done. > Loaded symbols for /boot/kernel/linux_common.ko > Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/linprocfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/linprocfs.ko > Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/tmpfs.ko > Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/fdescfs.ko > #0 doadump (textdump=0) at pcpu.h:221 > 221 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) #0 doadump (textdump=0) at pcpu.h:221 > #1 0x8037c6b6 in db_fncall (dummy1=, > dummy2=, dummy3=, > dummy4=) at /usr/src/sys/ddb/db_command.c:568 > #2 0x8037c14e in db_command (cmd_table=0x0) > at /usr/src/sys/ddb/db_command.c:440 > #3 0x8037bee4 in db_command_loop () > at /usr/src/sys/ddb/db_command.c:493 > #4 0x8037e97b in db_trap (type=, code=0) > at /usr/src/sys/ddb/db_main.c:251 > #5 0x80a5fc73 in kdb_trap (type=1, code=0, tf=) > at /usr/src/sys/kern/subr_kdb.c:654 > #6 0x80e69c81 in trap_fatal (frame=0xfe0859c63610, > eva=) at /usr/src/sys/amd64/amd64/trap.c:829 > #7 0x80e69951 in trap (frame=) > at /usr/src/sys/amd64/amd64/trap.c:203 > #8 0x80e498d7 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:234 > #9 0x8240fef5 in cpu_lock () from /boot/kernel/opensolaris.ko > #10 0xf804aaeb8000 in ?? () > #11 0x0001 in ?? () > #12 0xfe0859c63760 in ?? () > #13 0x82125325 in vdev_mirror_io_start (zio=0xf8047a6d9000) > at >
Re: 11.0-CURRENT r290039 privileged instruction fault while in kernel mode
On 28 Oct, Konstantin Belousov wrote: > On Tue, Oct 27, 2015 at 04:09:28PM -0700, Don Lewis wrote: >> I just got this crash while running poudriere on a freshly upgraded >> 11.0-CURRENT machine. The instruction pointer value looks pretty >> strange. >> >> >> FreeBSD zipper.catspoiler.org 11.0-CURRENT FreeBSD 11.0-CURRENT #30 r290039: >> Tue Oct 27 00:08:00 PDT 2015 >> d...@zipper.catspoiler.org:/usr/obj/usr/src/sys/GENERIC amd64 >> >> panic: >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you are >> welcome to change it and/or distribute copies of it under certain conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> >> >> Fatal trap 1: privileged instruction fault while in kernel mode >> cpuid = 4; apic id = 14 >> instruction pointer = 0x20:0x8240fef5 > What is the instruction at the reported address ? (kgdb) disassemble/r Dump of assembler code for function cpu_lock: 0x8240fef0 <+0>: 25 bb 40 82 ff and$0xff8240bb,%eax => 0x8240fef5 <+5>: ff (bad) 0x8240fef6 <+6>: ff (bad) 0x8240fef7 <+7>: ff 00 incl (%rax) 0x8240fef9 <+9>: 00 71 02add%dh,0x2(%rcx) 0x8240fefc <+12>:00 00 add%al,(%rax) 0x8240fefe <+14>:00 00 add%al,(%rax) 0x8240ff00 <+16>:00 00 add%al,(%rax) 0x8240ff02 <+18>:00 00 add%al,(%rax) 0x8240ff04 <+20>:00 00 add%al,(%rax) 0x8240ff06 <+22>:00 00 add%al,(%rax) 0x8240ff08 <+24>:01 00 add%eax,(%rax) 0x8240ff0a <+26>:00 00 add%al,(%rax) 0x8240ff0c <+28>:00 00 add%al,(%rax) 0x8240ff0e <+30>:00 00 add%al,(%rax) End of assembler dump. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: 11.0-CURRENT r290039 privileged instruction fault while in kernel mode
On Wed, Oct 28, 2015 at 10:16:12AM -0700, Don Lewis wrote: > On 28 Oct, Konstantin Belousov wrote: > > On Tue, Oct 27, 2015 at 04:09:28PM -0700, Don Lewis wrote: > >> I just got this crash while running poudriere on a freshly upgraded > >> 11.0-CURRENT machine. The instruction pointer value looks pretty > >> strange. > >> > >> > >> FreeBSD zipper.catspoiler.org 11.0-CURRENT FreeBSD 11.0-CURRENT #30 > >> r290039: Tue Oct 27 00:08:00 PDT 2015 > >> d...@zipper.catspoiler.org:/usr/obj/usr/src/sys/GENERIC amd64 > >> > >> panic: > >> > >> GNU gdb 6.1.1 [FreeBSD] > >> Copyright 2004 Free Software Foundation, Inc. > >> GDB is free software, covered by the GNU General Public License, and you > >> are > >> welcome to change it and/or distribute copies of it under certain > >> conditions. > >> Type "show copying" to see the conditions. > >> There is absolutely no warranty for GDB. Type "show warranty" for details. > >> This GDB was configured as "amd64-marcel-freebsd"... > >> > >> Unread portion of the kernel message buffer: > >> > >> > >> Fatal trap 1: privileged instruction fault while in kernel mode > >> cpuid = 4; apic id = 14 > >> instruction pointer= 0x20:0x8240fef5 > > What is the instruction at the reported address ? > > (kgdb) disassemble/r > Dump of assembler code for function cpu_lock: >0x8240fef0 <+0>: 25 bb 40 82 ff and$0xff8240bb,%eax > => 0x8240fef5 <+5>: ff (bad) >0x8240fef6 <+6>: ff (bad) >0x8240fef7 <+7>: ff 00 incl (%rax) >0x8240fef9 <+9>: 00 71 02add%dh,0x2(%rcx) >0x8240fefc <+12>: 00 00 add%al,(%rax) >0x8240fefe <+14>: 00 00 add%al,(%rax) >0x8240ff00 <+16>: 00 00 add%al,(%rax) >0x8240ff02 <+18>: 00 00 add%al,(%rax) >0x8240ff04 <+20>: 00 00 add%al,(%rax) >0x8240ff06 <+22>: 00 00 add%al,(%rax) >0x8240ff08 <+24>: 01 00 add%eax,(%rax) >0x8240ff0a <+26>: 00 00 add%al,(%rax) >0x8240ff0c <+28>: 00 00 add%al,(%rax) >0x8240ff0e <+30>: 00 00 add%al,(%rax) > End of assembler dump. Oh, I see. cpu_lock is mutex, dump above demonstrates is cleanly. Most likely, something overwrote some pointer to a function with the address. You probably have to bisect. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: 11.0-CURRENT r290039 privileged instruction fault while in kernel mode
On 28 Oct, Konstantin Belousov wrote: > On Wed, Oct 28, 2015 at 10:16:12AM -0700, Don Lewis wrote: >> On 28 Oct, Konstantin Belousov wrote: >> > On Tue, Oct 27, 2015 at 04:09:28PM -0700, Don Lewis wrote: >> >> I just got this crash while running poudriere on a freshly upgraded >> >> 11.0-CURRENT machine. The instruction pointer value looks pretty >> >> strange. >> >> >> >> >> >> FreeBSD zipper.catspoiler.org 11.0-CURRENT FreeBSD 11.0-CURRENT #30 >> >> r290039: Tue Oct 27 00:08:00 PDT 2015 >> >> d...@zipper.catspoiler.org:/usr/obj/usr/src/sys/GENERIC amd64 >> >> >> >> panic: >> >> >> >> GNU gdb 6.1.1 [FreeBSD] >> >> Copyright 2004 Free Software Foundation, Inc. >> >> GDB is free software, covered by the GNU General Public License, and you >> >> are >> >> welcome to change it and/or distribute copies of it under certain >> >> conditions. >> >> Type "show copying" to see the conditions. >> >> There is absolutely no warranty for GDB. Type "show warranty" for >> >> details. >> >> This GDB was configured as "amd64-marcel-freebsd"... >> >> >> >> Unread portion of the kernel message buffer: >> >> >> >> >> >> Fatal trap 1: privileged instruction fault while in kernel mode >> >> cpuid = 4; apic id = 14 >> >> instruction pointer = 0x20:0x8240fef5 >> > What is the instruction at the reported address ? >> >> (kgdb) disassemble/r >> Dump of assembler code for function cpu_lock: >>0x8240fef0 <+0>: 25 bb 40 82 ff and$0xff8240bb,%eax >> => 0x8240fef5 <+5>: ff (bad) >>0x8240fef6 <+6>: ff (bad) >>0x8240fef7 <+7>: ff 00 incl (%rax) >>0x8240fef9 <+9>: 00 71 02add%dh,0x2(%rcx) >>0x8240fefc <+12>: 00 00 add%al,(%rax) >>0x8240fefe <+14>: 00 00 add%al,(%rax) >>0x8240ff00 <+16>: 00 00 add%al,(%rax) >>0x8240ff02 <+18>: 00 00 add%al,(%rax) >>0x8240ff04 <+20>: 00 00 add%al,(%rax) >>0x8240ff06 <+22>: 00 00 add%al,(%rax) >>0x8240ff08 <+24>: 01 00 add%eax,(%rax) >>0x8240ff0a <+26>: 00 00 add%al,(%rax) >>0x8240ff0c <+28>: 00 00 add%al,(%rax) >>0x8240ff0e <+30>: 00 00 add%al,(%rax) >> End of assembler dump. > > Oh, I see. cpu_lock is mutex, dump above demonstrates is cleanly. > Most likely, something overwrote some pointer to a function with > the address. > > You probably have to bisect. The could be difficult. Whatever this is, it seems to be very hard to trigger. The machine was up and doing a lot of poudriere package building for about a day before it crashed. It's now got close to a day of uptime again, mostly building packages, without another crash. The previous kernel was r289123. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
11.0-CURRENT r290039 privileged instruction fault while in kernel mode
I just got this crash while running poudriere on a freshly upgraded 11.0-CURRENT machine. The instruction pointer value looks pretty strange. FreeBSD zipper.catspoiler.org 11.0-CURRENT FreeBSD 11.0-CURRENT #30 r290039: Tue Oct 27 00:08:00 PDT 2015 d...@zipper.catspoiler.org:/usr/obj/usr/src/sys/GENERIC amd64 panic: GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 1: privileged instruction fault while in kernel mode cpuid = 4; apic id = 14 instruction pointer = 0x20:0x8240fef5 stack pointer = 0x28:0xfe0859c636c0 frame pointer = 0x28:0xfe0859c636e0 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 0 (zio_free_issue_2_9) Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/modules/vboxdrv.ko...done. Loaded symbols for /boot/modules/vboxdrv.ko Reading symbols from /boot/modules/vboxnetflt.ko...done. Loaded symbols for /boot/modules/vboxnetflt.ko Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /usr/lib/debug//boot/kernel/netgraph.ko.debug...done. done. Loaded symbols for /boot/kernel/netgraph.ko Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from /usr/lib/debug//boot/kernel/ng_ether.ko.debug...done. done. Loaded symbols for /boot/kernel/ng_ether.ko Reading symbols from /boot/modules/vboxnetadp.ko...done. Loaded symbols for /boot/modules/vboxnetadp.ko Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/nullfs.ko.debug...done. done. Loaded symbols for /boot/kernel/nullfs.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux.ko.debug...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/linux_common.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux_common.ko.debug...done. done. Loaded symbols for /boot/kernel/linux_common.ko Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/linprocfs.ko.debug...done. done. Loaded symbols for /boot/kernel/linprocfs.ko Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done. done. Loaded symbols for /boot/kernel/tmpfs.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. done. Loaded symbols for /boot/kernel/fdescfs.ko #0 doadump (textdump=0) at pcpu.h:221 221 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=0) at pcpu.h:221 #1 0x8037c6b6 in db_fncall (dummy1=, dummy2=, dummy3=, dummy4=) at /usr/src/sys/ddb/db_command.c:568 #2 0x8037c14e in db_command (cmd_table=0x0) at /usr/src/sys/ddb/db_command.c:440 #3 0x8037bee4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493 #4 0x8037e97b in db_trap (type=, code=0) at /usr/src/sys/ddb/db_main.c:251 #5 0x80a5fc73 in kdb_trap (type=1, code=0, tf=) at /usr/src/sys/kern/subr_kdb.c:654 #6 0x80e69c81 in trap_fatal (frame=0xfe0859c63610, eva=) at /usr/src/sys/amd64/amd64/trap.c:829 #7 0x80e69951 in trap (frame=) at /usr/src/sys/amd64/amd64/trap.c:203 #8 0x80e498d7 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:234 #9 0x8240fef5 in cpu_lock () from /boot/kernel/opensolaris.ko #10 0xf804aaeb8000 in ?? () #11 0x0001 in ?? () #12 0xfe0859c63760 in ?? () #13 0x82125325 in vdev_mirror_io_start (zio=0xf8047a6d9000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c:475 Previous frame identical to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to
