Re: watchdogd coredump
On Sun, 4 Nov 2012 00:08:43 +0200 Konstantin Belousov kostik...@gmail.com wrote: On Sat, Nov 03, 2012 at 10:55:42PM +0100, Alexander Leidinger wrote: Hi, I updated from r239708 to r242511 and my watchdogd coredumps (and brings down the system... well, the WD works). Before I have a deeper look (recompiling with debugging and such) at this, can someone confirm that with a recent -current and a WD configured to call a shell script which does a simple ls to /dev/null there is a segfault in watchdogd? watchdogd_flags=-e /root/bin/wd_check.sh -s 5 -t 60 ---snip--- # cat /root/bin/wd_check.sh #!/bin/sh exec ls / /space/jails /dev/null 21 /dev/null ---snip--- The not so useful backtrace of the watchdogd core: ---snip--- #0 0x000800609520 in dlopen () from /libexec/ld-elf.so.1 #1 0x00080060309d in .text () from /libexec/ld-elf.so.1 #2 0x0246 in ?? () #3 0xfe001ee700f8 in ?? () #4 0xfe001ee700f8 in ?? () #5 0x80647e80 in ?? () #6 0x0001 in ?? () #7 0x7fffdb90 in ?? () #8 0x0013 in ?? () #9 0x7fffdb80 in ?? () #10 0x in ?? () #11 0x0206 in ?? () #12 0x7fffdaf0 in ?? () #13 0x00080061cc00 in ?? () #14 0x0031 in ?? () #15 0x000800c93dc9 in system () from /lib/libc.so.7 #16 0x00401558 in ?? () #17 0x00400f7e in ?? () #18 0x00080061c000 in ?? () #19 0x in ?? () #20 0x0007 in ?? () #21 0x7fffdef8 in ?? () ---snip--- Are you sure that your kernel is at r242511 ? The issue should have been fixed by r242011. I got some time again to look into this. I recompiled the libs with debugging symbols and installed unstripped versions, but I don't get an useful backtrace: ---snip--- # gdb /usr/sbin/watchdogd /watchdogd.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as amd64-marcel-freebsd...(no debugging symbols found)... Core was generated by `watchdogd'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libm.so.5...done. Loaded symbols for /lib/libm.so.5 Reading symbols from /lib/libutil.so.9...done. Loaded symbols for /lib/libutil.so.9 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x000800609520 in dlopen () from /libexec/ld-elf.so.1 (gdb) bt #0 0x000800609520 in dlopen () from /libexec/ld-elf.so.1 #1 0x00080060309d in .text () from /libexec/ld-elf.so.1 #2 0x0246 in ?? () #3 0xfe0179f060f8 in ?? () #4 0xfe0179f060f8 in ?? () #5 0x80647e80 in ?? () #6 0x0001 in ?? () #7 0x7fffdb90 in ?? () #8 0x0013 in ?? () #9 0x7fffdb80 in ?? () #10 0x in ?? () #11 0x0206 in ?? () #12 0x7fffdaf0 in ?? () #13 0x00080061cc00 in ?? () #14 0x0031 in ?? () #15 0x000800c93dc9 in __system ( command=0x801406040 /root/bin/wd_check.sh) at /space/system/usr_src/lib/libc/stdlib/system.c:70 #16 0x00401558 in main () # uname -v FreeBSD 10.0-CURRENT #10 r242511M: Sat Nov 3 17:49:09 CET 2012 ---snip--- How can I debug this further? Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: watchdogd coredump
On Sat, 03 Nov 2012 15:47:39 -0700 Xin Li delp...@delphij.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 11/3/12 3:34 PM, Alexander Leidinger wrote: On Sun, 4 Nov 2012 00:08:43 +0200 Konstantin Belousov kostik...@gmail.com wrote: Are you sure that your kernel is at r242511 ? The issue should have been fixed by r242011. # svnversion 242511M # svn status M contrib/bind9/bin/named/interfacemgr.c M etc/defaults/rc.conf M etc/rc.d/jail M sys/dev/ata/ata-all.c M sys/dev/drm/drmP.h M sys/dev/usb/serial/ulpt.c M sys/kern/kern_jail.c M sys/sys/jail.h M sys/sys/priv.h M sys/vm/uma_core.c The uma_core patch is the one floating around which shall prevent memory fragmentation, the others are mostly my X-in-jail and some minor default values (ata, ulpt) changes. Currently I'm back to the previous kernel+world, but I still have the r242511M boot environment available (I have no time to investigate the problem this evening, and I want to have a stable system until I get time to have a deeper look at this). What was 'strings /boot/kernel.bad/kernel | tail' saying? I verified at boot that the kernel was booting the right kernel: ---snip--- # strings /rpool/ROOT/r242511M/boot/kernel/kernel | tail [...] @(#)FreeBSD 10.0-CURRENT #10 r242511M: Sat Nov 3 17:49:09 CET 2012 FreeBSD 10.0-CURRENT #10 r242511M: Sat Nov 3 17:49:09 CET 2012 r...@andromeda.leidinger.net:/usr/obj/space/system/usr_src/sys/ANDROMEDA [...] ---snip--- Anything special you want me to investigate? Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
watchdogd coredump
Hi, I updated from r239708 to r242511 and my watchdogd coredumps (and brings down the system... well, the WD works). Before I have a deeper look (recompiling with debugging and such) at this, can someone confirm that with a recent -current and a WD configured to call a shell script which does a simple ls to /dev/null there is a segfault in watchdogd? watchdogd_flags=-e /root/bin/wd_check.sh -s 5 -t 60 ---snip--- # cat /root/bin/wd_check.sh #!/bin/sh exec ls / /space/jails /dev/null 21 /dev/null ---snip--- The not so useful backtrace of the watchdogd core: ---snip--- #0 0x000800609520 in dlopen () from /libexec/ld-elf.so.1 #1 0x00080060309d in .text () from /libexec/ld-elf.so.1 #2 0x0246 in ?? () #3 0xfe001ee700f8 in ?? () #4 0xfe001ee700f8 in ?? () #5 0x80647e80 in ?? () #6 0x0001 in ?? () #7 0x7fffdb90 in ?? () #8 0x0013 in ?? () #9 0x7fffdb80 in ?? () #10 0x in ?? () #11 0x0206 in ?? () #12 0x7fffdaf0 in ?? () #13 0x00080061cc00 in ?? () #14 0x0031 in ?? () #15 0x000800c93dc9 in system () from /lib/libc.so.7 #16 0x00401558 in ?? () #17 0x00400f7e in ?? () #18 0x00080061c000 in ?? () #19 0x in ?? () #20 0x0007 in ?? () #21 0x7fffdef8 in ?? () ---snip--- Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: watchdogd coredump
On Sat, Nov 03, 2012 at 10:55:42PM +0100, Alexander Leidinger wrote: Hi, I updated from r239708 to r242511 and my watchdogd coredumps (and brings down the system... well, the WD works). Before I have a deeper look (recompiling with debugging and such) at this, can someone confirm that with a recent -current and a WD configured to call a shell script which does a simple ls to /dev/null there is a segfault in watchdogd? watchdogd_flags=-e /root/bin/wd_check.sh -s 5 -t 60 ---snip--- # cat /root/bin/wd_check.sh #!/bin/sh exec ls / /space/jails /dev/null 21 /dev/null ---snip--- The not so useful backtrace of the watchdogd core: ---snip--- #0 0x000800609520 in dlopen () from /libexec/ld-elf.so.1 #1 0x00080060309d in .text () from /libexec/ld-elf.so.1 #2 0x0246 in ?? () #3 0xfe001ee700f8 in ?? () #4 0xfe001ee700f8 in ?? () #5 0x80647e80 in ?? () #6 0x0001 in ?? () #7 0x7fffdb90 in ?? () #8 0x0013 in ?? () #9 0x7fffdb80 in ?? () #10 0x in ?? () #11 0x0206 in ?? () #12 0x7fffdaf0 in ?? () #13 0x00080061cc00 in ?? () #14 0x0031 in ?? () #15 0x000800c93dc9 in system () from /lib/libc.so.7 #16 0x00401558 in ?? () #17 0x00400f7e in ?? () #18 0x00080061c000 in ?? () #19 0x in ?? () #20 0x0007 in ?? () #21 0x7fffdef8 in ?? () ---snip--- Are you sure that your kernel is at r242511 ? The issue should have been fixed by r242011. pgpmusxssaiV7.pgp Description: PGP signature
Re: watchdogd coredump
On Sat, Nov 3, 2012 at 2:55 PM, Alexander Leidinger alexan...@leidinger.net wrote: Hi, I updated from r239708 to r242511 and my watchdogd coredumps (and brings down the system... well, the WD works). Before I have a deeper look (recompiling with debugging and such) at this, can someone confirm that with a recent -current and a WD configured to call a shell script which does a simple ls to /dev/null there is a segfault in watchdogd? watchdogd_flags=-e /root/bin/wd_check.sh -s 5 -t 60 ---snip--- # cat /root/bin/wd_check.sh #!/bin/sh exec ls / /space/jails /dev/null 21 /dev/null ---snip--- The not so useful backtrace of the watchdogd core: ---snip--- #0 0x000800609520 in dlopen () from /libexec/ld-elf.so.1 #1 0x00080060309d in .text () from /libexec/ld-elf.so.1 #2 0x0246 in ?? () #3 0xfe001ee700f8 in ?? () #4 0xfe001ee700f8 in ?? () #5 0x80647e80 in ?? () #6 0x0001 in ?? () #7 0x7fffdb90 in ?? () #8 0x0013 in ?? () #9 0x7fffdb80 in ?? () #10 0x in ?? () #11 0x0206 in ?? () #12 0x7fffdaf0 in ?? () #13 0x00080061cc00 in ?? () #14 0x0031 in ?? () #15 0x000800c93dc9 in system () from /lib/libc.so.7 #16 0x00401558 in ?? () #17 0x00400f7e in ?? () #18 0x00080061c000 in ?? () #19 0x in ?? () #20 0x0007 in ?? () #21 0x7fffdef8 in ?? () ---snip--- Do you have watchdogd_flags set to something non-standard? Xin CCed since he made the last commit. -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: watchdogd coredump
On Sat, 3 Nov 2012 15:13:59 -0700 Garrett Cooper yaneg...@gmail.com wrote: On Sat, Nov 3, 2012 at 2:55 PM, Alexander Leidinger alexan...@leidinger.net wrote: Hi, I updated from r239708 to r242511 and my watchdogd coredumps (and brings down the system... well, the WD works). Before I have a deeper look (recompiling with debugging and such) at this, can someone confirm that with a recent -current and a WD configured to call a shell script which does a simple ls to /dev/null there is a segfault in watchdogd? watchdogd_flags=-e /root/bin/wd_check.sh -s 5 -t 60 ---snip--- # cat /root/bin/wd_check.sh #!/bin/sh exec ls / /space/jails /dev/null 21 /dev/null ---snip--- Do you have watchdogd_flags set to something non-standard? Xin CCed since he made the last commit. Please see above. Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: watchdogd coredump
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 11/3/12 3:13 PM, Garrett Cooper wrote: On Sat, Nov 3, 2012 at 2:55 PM, Alexander Leidinger alexan...@leidinger.net mailto:alexan...@leidinger.net wrote: Hi, I updated from r239708 to r242511 and my watchdogd coredumps (and brings down the system... well, the WD works). Before I have a deeper look (recompiling with debugging and such) at this, can someone confirm that with a recent -current and a WD configured to call a shell script which does a simple ls to /dev/null there is a segfault in watchdogd? watchdogd_flags=-e /root/bin/wd_check.sh -s 5 -t 60 [...] ^^ Do you have watchdogd_flags set to something non-standard? Xin CCed since he made the last commit. I think my commit have nothing to do with Alexander's issue, it's fairly self-contained with no outreaching memory access that could cause access violation. I think Kostik was right that he may be running an older version of kernel with watchdog newer than 239896. Alexander, can you confirm if that was the case? Cheers, -BEGIN PGP SIGNATURE- iQEcBAEBCAAGBQJQlZvBAAoJEG80Jeu8UPuzmXIH/j1w4BFxPEfkj4nipSurLq/n 3jLExTaGho4ElO82HjAx6hK0BO5aLHKG5A5FrLJaD5oH61xqDMg0vjcbtausLUJs Pknnf91UtWuyZ3odvcrY0Y7Uv1flQvBri6ZpsmJXCqMQvNh2Uks4+0iNi6vRs8dN eh3igb6tGt+arOwcUohUo60sivTUPl3KVOSRvZlxAuCzrTGwwJ3B2wDSI2aBI34B qdfUFB1XcZwalCCbI9opKYnVdQOFrTbypVB0aV38IYkRNLPIbKzvDwHbNTA3N8Fz wHe3qMiN+De23X19sNvfKHDBZtaz3LFfTnyw4eDkaxocEVscUgeSZxJdhDSBaHg= =ar5w -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: watchdogd coredump
On Sun, 4 Nov 2012 00:08:43 +0200 Konstantin Belousov kostik...@gmail.com wrote: Are you sure that your kernel is at r242511 ? The issue should have been fixed by r242011. # svnversion 242511M # svn status M contrib/bind9/bin/named/interfacemgr.c M etc/defaults/rc.conf M etc/rc.d/jail M sys/dev/ata/ata-all.c M sys/dev/drm/drmP.h M sys/dev/usb/serial/ulpt.c M sys/kern/kern_jail.c M sys/sys/jail.h M sys/sys/priv.h M sys/vm/uma_core.c The uma_core patch is the one floating around which shall prevent memory fragmentation, the others are mostly my X-in-jail and some minor default values (ata, ulpt) changes. Currently I'm back to the previous kernel+world, but I still have the r242511M boot environment available (I have no time to investigate the problem this evening, and I want to have a stable system until I get time to have a deeper look at this). Bye, Alexander. -- http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: watchdogd coredump
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 11/3/12 3:34 PM, Alexander Leidinger wrote: On Sun, 4 Nov 2012 00:08:43 +0200 Konstantin Belousov kostik...@gmail.com wrote: Are you sure that your kernel is at r242511 ? The issue should have been fixed by r242011. # svnversion 242511M # svn status M contrib/bind9/bin/named/interfacemgr.c M etc/defaults/rc.conf M etc/rc.d/jail M sys/dev/ata/ata-all.c M sys/dev/drm/drmP.h M sys/dev/usb/serial/ulpt.c M sys/kern/kern_jail.c M sys/sys/jail.h M sys/sys/priv.h M sys/vm/uma_core.c The uma_core patch is the one floating around which shall prevent memory fragmentation, the others are mostly my X-in-jail and some minor default values (ata, ulpt) changes. Currently I'm back to the previous kernel+world, but I still have the r242511M boot environment available (I have no time to investigate the problem this evening, and I want to have a stable system until I get time to have a deeper look at this). What was 'strings /boot/kernel.bad/kernel | tail' saying? Cheers, -BEGIN PGP SIGNATURE- iQEcBAEBCAAGBQJQlZ8LAAoJEG80Jeu8UPuz2S8IAKfvH1m7L3TQay3tghddaI1k ohMic676FX/u24lxFpO9ENPxZq28VKyTIKG+5XD2dxfYvX9kRRmfRUVQHHc5Ri2H c/ZPusPFkOCS3/U74puHvgIF+ypi7110AOC4S+T6whW9D8SVLL/Hauu05CbbjYb1 ef5Vhj0xnxa+XlWhDY6h4QPeduvzulrxhcjJiyiS5aH+ZBMph26cBUfxyrzfMV+2 akJbI0KV0z0AR5HgwS16CLVahApqEuRyXWNmwEuE2c234q0clXarsJS7biDy0X2P eeFogNkoOJzVbLVOmKtgG1l63yEfAKtDTJURtaOTB7/COG4KkeWolj3elL60MVw= =a1YO -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
