Re: dhclient wedged
On Thu, Feb 02, 2006 at 10:05:44PM -0800, John T. Yocum wrote: I've been having a same sort of issue that others have reported. After dhclient has been running for a random length of time, it suddenly starts consuming a large amount of CPU. Unfortunately for me, the longest it tends to run is 48 hours before acting up. I did get it rebuilt with debug symbols, and got a stack trace from it, while it was acting up. Hopefully, it will be useful. I really need a core dump. A single stack trace just tells me the packet processing routing isn't working (or at least everything is getting rejected). Unfortunatly, I haven't had any luck finding bugs there by inspection. -- Brooks -- Any statement of the form X is the one, true Y is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 pgp6aMMERprXM.pgp Description: PGP signature
dhclient wedged
I've been having a same sort of issue that others have reported. After dhclient has been running for a random length of time, it suddenly starts consuming a large amount of CPU. Unfortunately for me, the longest it tends to run is 48 hours before acting up. I did get it rebuilt with debug symbols, and got a stack trace from it, while it was acting up. Hopefully, it will be useful. Attaching to program: /sbin/dhclient, process 2248 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 0x08051a0d in receive_packet (interface=0x8071000, buf=0xbfbfdc20 \002\001\006, len=4096, from=0xbfbfec70, hfrom=0xbfbfec50) at /usr/src/sbin/dhclient/bpf.c:308 308 memcpy(hdr, interface-rbuf[interface-rbuf_offset], (gdb) bt #0 0x08051a0d in receive_packet (interface=0x8071000, buf=0xbfbfdc20 \002\001\006, len=4096, from=0xbfbfec70, hfrom=0xbfbfec50) at /usr/src/sbin/dhclient/bpf.c:308 #1 0x08050dc1 in got_one (l=0x80741b0) at /usr/src/sbin/dhclient/dispatch.c:256 #2 0x08050d58 in dispatch () at /usr/src/sbin/dhclient/dispatch.c:228 #3 0x0804a0d0 in main (argc=1, argv=0xbfbfed3c) at /usr/src/sbin/dhclient/dhclient.c:429 BTW: For reference I'm running 6.0-RELEASE-p4. Thanks, John ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: dhclient wedged
On Tue, 2006-Jan-31 15:38:53 -0500, Lodewijk Vge wrote: On 31-jan-2006, at 14:13, Brooks Davis wrote: At the very least I need a coredump and your executable so I can look at variables in receive_packet. I accidentally killed it with my attempts to make it dump a core file. so, what should I do the next time it happens to make it dump core? kill -QUIT ... is the generic answer. sigaction(2) provides the definitive list of which signals default to dumping core. -- Peter Jeremy ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: dhclient wedged
On 28-jan-2006, at 20:33, Kelly D. Grills wrote: Just a quick update. I've recompiled and am waiting for a failure. If I recall correctly my problem occured when the ip actually changed, so it may be a while before I post any debugging info. it just wedged on my machine, also on comcast. this is the backtrace when I attach gdb: (gdb) bt #0 0x2813929a in memcpy () from /lib/libc.so.6 #1 0x0805199d in receive_packet (interface=0x807, buf=0xbfbfdb60 \002\001\006, len=4096, from=0xbfbfebb0, hfrom=0xbfbfeb90) at /usr/src/sbin/dhclient/bpf.c:306 #2 0x08050d51 in got_one (l=0x8073230) at /usr/src/sbin/dhclient/dispatch.c:256 #3 0x08050ce8 in dispatch () at /usr/src/sbin/dhclient/dispatch.c:228 #4 0x0804a01c in main (argc=1, argv=0xbfbfec98) at /usr/src/sbin/dhclient/dhclient.c:429 Lodewijk ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: dhclient wedged
On Tue, Jan 31, 2006 at 12:29:48PM -0500, Lodewijk Vöge wrote: On 28-jan-2006, at 20:33, Kelly D. Grills wrote: Just a quick update. I've recompiled and am waiting for a failure. If I recall correctly my problem occured when the ip actually changed, so it may be a while before I post any debugging info. it just wedged on my machine, also on comcast. this is the backtrace when I attach gdb: (gdb) bt #0 0x2813929a in memcpy () from /lib/libc.so.6 #1 0x0805199d in receive_packet (interface=0x807, buf=0xbfbfdb60 \002\001\006, len=4096, from=0xbfbfebb0, hfrom=0xbfbfeb90) at /usr/src/sbin/dhclient/bpf.c:306 #2 0x08050d51 in got_one (l=0x8073230) at /usr/src/sbin/dhclient/dispatch.c:256 #3 0x08050ce8 in dispatch () at /usr/src/sbin/dhclient/dispatch.c:228 #4 0x0804a01c in main (argc=1, argv=0xbfbfec98) at /usr/src/sbin/dhclient/dhclient.c:429 The backtrace doesn't tell me much at first glance. I'm quite certain it's not hung in memcpy since I'm highly confident memcpy doesn't have an undiscovered bugs. :) Instead it's presumably looping in receive_packet or at a higher level, possibaly in dispatch. At the very least I need a coredump and your executable so I can look at variables in receive_packet. Thanks for your report. -- Brooks -- Any statement of the form X is the one, true Y is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 pgp3EEkAqmRuE.pgp Description: PGP signature
Re: dhclient wedged
On 31-jan-2006, at 14:13, Brooks Davis wrote: At the very least I need a coredump and your executable so I can look at variables in receive_packet. I accidentally killed it with my attempts to make it dump a core file. so, what should I do the next time it happens to make it dump core? Lodewijk ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: dhclient wedged
On Sun, Jan 22, 2006 at 06:40:19PM -0600, Kelly D. Grills wrote: On Sun, Jan 22, 2006 at 02:18:28PM -0800, Brooks Davis wrote: This definitly sounds like something particular to your dhcp servers. It would be nice if we could fix it, but without some debugging help that's going to be pretty much impossible. If you can recompile dhclient with debugging symbols and when it goes into a spin, use the debugger to figure out where it's spinning, that would be useful. See this post for details: http://lists.freebsd.org/pipermail/freebsd-current/2005-August/054224.html I'm recompiling at the moment. Will follow up with debugging results when it fails. Just a quick update. I've recompiled and am waiting for a failure. If I recall correctly my problem occured when the ip actually changed, so it may be a while before I post any debugging info. -- Kelly D. Grills [EMAIL PROTECTED] pgpC24knsJv7G.pgp Description: PGP signature
Re: dhclient wedged
Kelly, On Sun, 22 Jan 2006, Kelly D. Grills wrote: =On Sun, Jan 22, 2006 at 02:52:44AM -0500, Forrest Aldrich wrote: = = I reported this problem a few OS versions back... pre-6. I'm on = FreeBSD-6-STABLE, and I've found the dhclient once again wedged in a = mode that was eating a lot of CPU. = = The solution is to kill it, and restart. = = I'm on Comcast's network, so I don't really know if their DHCP server is = doing something that FreeBSD's stock dhclient doesn't like; however, I = wonder if someone else has noticed this problem, etc. = =Yes, I've also had the same problem (6.0-RELEASE), on Comcast also. =I installed isc-dhcp3-client-3.0.3_1 from ports and haven't had a =problem since. (crossing fingers ;=) not sure if it's the same problem for you, but Comcast in my area supplies a bad entry for DNS Suffix/Search order ((hsd1.ca.comcast.net)(hsd2.ca.comcast.net)) And my little netgear router doesn't like that entry and _seems_ to only supply hsd1.ca.comcast.net)) via dhcp. My Mac and Winbloze boxes do fine, but FBSD rejects the dhoffer unless isc's dhclient is used. I haven't tried plugging directly into the Comcast router, so I don't know if FBSD will accept the original offer with parens... Thanks, Will -- Will Froning Unix Sys. Admin. [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: dhclient wedged
On Fri, Jan 27, 2006 at 06:30:36AM -0800, Will Froning wrote: Kelly, On Sun, 22 Jan 2006, Kelly D. Grills wrote: =On Sun, Jan 22, 2006 at 02:52:44AM -0500, Forrest Aldrich wrote: = = I reported this problem a few OS versions back... pre-6. I'm on = FreeBSD-6-STABLE, and I've found the dhclient once again wedged in a = mode that was eating a lot of CPU. = = The solution is to kill it, and restart. = = I'm on Comcast's network, so I don't really know if their DHCP server is = doing something that FreeBSD's stock dhclient doesn't like; however, I = wonder if someone else has noticed this problem, etc. = =Yes, I've also had the same problem (6.0-RELEASE), on Comcast also. =I installed isc-dhcp3-client-3.0.3_1 from ports and haven't had a =problem since. (crossing fingers ;=) not sure if it's the same problem for you, but Comcast in my area supplies a bad entry for DNS Suffix/Search order ((hsd1.ca.comcast.net)(hsd2.ca.comcast.net)) I must say that's a really creative one. I wonder if it even works for Windows. And my little netgear router doesn't like that entry and _seems_ to only supply hsd1.ca.comcast.net)) via dhcp. My Mac and Winbloze boxes do fine, but FBSD rejects the dhoffer unless isc's dhclient is used. I haven't tried plugging directly into the Comcast router, so I don't know if FBSD will accept the original offer with parens... If you upgrade to 6.0-STABLE the offer won't be rejected, it will just delete the invalid entry. -- Brooks -- Any statement of the form X is the one, true Y is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 pgpRBidrMG5pi.pgp Description: PGP signature
Re: dhclient wedged
Brooks Davis [EMAIL PROTECTED] wrote: This definitly sounds like something particular to your dhcp servers. It would be nice if we could fix it, but without some debugging help that's going to be pretty much impossible. [...] It happens to me quite often, too. The only thing related in the non-debug messages is: Jan 8 12:02:19 moneypenny dhclient[68091]: 5 bad IP checksums seen in 5 packets Jan 8 12:02:49 moneypenny last message repeated 743054 times Jan 8 12:04:50 moneypenny last message repeated 2951866 times Jan 8 12:14:51 moneypenny last message repeated 14457921 times Jan 8 12:24:52 moneypenny last message repeated 14812032 times Jan 8 12:34:53 moneypenny last message repeated 14770327 times Jan 8 12:44:55 moneypenny last message repeated 14748300 times Jan 8 12:51:44 moneypenny last message repeated 10037074 times ... which accounts for the CPU usage, I guess. I killed the bad IP checksums messages, so it doesn't annoy my syslog anymore, but it of course didn't fix the underlaying issue. I was looking at those packets with tcpdump once and didn't see anything obvious/bad there. And yes, I didn't have this problem with ISC client. And I surely use different cable provider, than the original poster ;) -- Krzysztof Kowalik | () ASCII Ribbon Campaign Computer Center, AGH UST| /\ Support plain text e-mail ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: dhclient wedged
On Mon, Jan 23, 2006 at 10:06:56AM +0100, Krzysztof Kowalik wrote: Brooks Davis [EMAIL PROTECTED] wrote: This definitly sounds like something particular to your dhcp servers. It would be nice if we could fix it, but without some debugging help that's going to be pretty much impossible. [...] It happens to me quite often, too. The only thing related in the non-debug messages is: Jan 8 12:02:19 moneypenny dhclient[68091]: 5 bad IP checksums seen in 5 packets Jan 8 12:02:49 moneypenny last message repeated 743054 times Jan 8 12:04:50 moneypenny last message repeated 2951866 times Jan 8 12:14:51 moneypenny last message repeated 14457921 times Jan 8 12:24:52 moneypenny last message repeated 14812032 times Jan 8 12:34:53 moneypenny last message repeated 14770327 times Jan 8 12:44:55 moneypenny last message repeated 14748300 times Jan 8 12:51:44 moneypenny last message repeated 10037074 times ... which accounts for the CPU usage, I guess. I killed the bad IP checksums messages, so it doesn't annoy my syslog anymore, but it of course didn't fix the underlaying issue. I was looking at those packets with tcpdump once and didn't see anything obvious/bad there. And yes, I didn't have this problem with ISC client. And I surely use different cable provider, than the original poster ;) What NIC are you using? This particular issue sounds like a NIC returning corrupt packets for some reason. Alternatively, the sending server could be producing corrupt packets. Some tcpdump traces (preferably raw dumps) could be useful. -- Brooks -- Any statement of the form X is the one, true Y is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 pgprpjH6da2ZF.pgp Description: PGP signature
Re: dhclient wedged
Brooks Davis [EMAIL PROTECTED] wrote: What NIC are you using? fxp0: Intel 82559 Pro/100 Ethernet port 0xc000-0xc03f mem 0xdb121000-0xdb121fff,0xdb00-0xdb0f irq 11 at device 1.0 on pci2 This particular issue sounds like a NIC returning corrupt packets for some reason. Alternatively, the sending server could be producing corrupt packets. Some tcpdump traces (preferably raw dumps) could be useful. As I said, I didn't see anything bad in the tcpdump. And since I really didn't have such issues before, and I don't see anything weird in any other place... I recompiled dhclient with -g and will tcpdump the traffic again, if I notice it eating 99% of CPU time again. -- Krzysztof Kowalik | () ASCII Ribbon Campaign Computer Center, AGH UST| /\ Support plain text e-mail ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: dhclient wedged
On Mon, Jan 23, 2006 at 11:25:25PM +0100, Krzysztof Kowalik wrote: Brooks Davis [EMAIL PROTECTED] wrote: What NIC are you using? fxp0: Intel 82559 Pro/100 Ethernet port 0xc000-0xc03f mem 0xdb121000-0xdb121fff,0xdb00-0xdb0f irq 11 at device 1.0 on pci2 What's ifconfig say? This particular issue sounds like a NIC returning corrupt packets for some reason. Alternatively, the sending server could be producing corrupt packets. Some tcpdump traces (preferably raw dumps) could be useful. As I said, I didn't see anything bad in the tcpdump. And since I really didn't have such issues before, and I don't see anything weird in any other place... I recompiled dhclient with -g and will tcpdump the traffic again, if I notice it eating 99% of CPU time again. OK, thanks. My other guess would be that we're getting misaligned relative to the buffer somehow and that's causing all the checksums to fail. If you can get into gdb when dhclient is spinning and check the values of interface-rbuf_offset, interface-rbuf_len, and length in receive_packet() that might tell us something intresting. -- Brooks -- Any statement of the form X is the one, true Y is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 pgpEH1LOU5INk.pgp Description: PGP signature
Re: dhclient wedged
On Sun, Jan 22, 2006 at 02:52:44AM -0500, Forrest Aldrich wrote: I reported this problem a few OS versions back... pre-6. I'm on FreeBSD-6-STABLE, and I've found the dhclient once again wedged in a mode that was eating a lot of CPU. The solution is to kill it, and restart. I'm on Comcast's network, so I don't really know if their DHCP server is doing something that FreeBSD's stock dhclient doesn't like; however, I wonder if someone else has noticed this problem, etc. Yes, I've also had the same problem (6.0-RELEASE), on Comcast also. I installed isc-dhcp3-client-3.0.3_1 from ports and haven't had a problem since. (crossing fingers ;=) -- Kelly D. Grills [EMAIL PROTECTED] pgpDlQTBKxIUa.pgp Description: PGP signature
Re: dhclient wedged
On Sun, Jan 22, 2006 at 09:51:43AM -0600, Kelly D. Grills wrote: On Sun, Jan 22, 2006 at 02:52:44AM -0500, Forrest Aldrich wrote: I reported this problem a few OS versions back... pre-6. I'm on FreeBSD-6-STABLE, and I've found the dhclient once again wedged in a mode that was eating a lot of CPU. The solution is to kill it, and restart. I'm on Comcast's network, so I don't really know if their DHCP server is doing something that FreeBSD's stock dhclient doesn't like; however, I wonder if someone else has noticed this problem, etc. Yes, I've also had the same problem (6.0-RELEASE), on Comcast also. I installed isc-dhcp3-client-3.0.3_1 from ports and haven't had a problem since. (crossing fingers ;=) This definitly sounds like something particular to your dhcp servers. It would be nice if we could fix it, but without some debugging help that's going to be pretty much impossible. If you can recompile dhclient with debugging symbols and when it goes into a spin, use the debugger to figure out where it's spinning, that would be useful. See this post for details: http://lists.freebsd.org/pipermail/freebsd-current/2005-August/054224.html -- Brooks -- Any statement of the form X is the one, true Y is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 pgp5Iv4wgDxY6.pgp Description: PGP signature
Re: dhclient wedged
On Sun, Jan 22, 2006 at 02:18:28PM -0800, Brooks Davis wrote: This definitly sounds like something particular to your dhcp servers. It would be nice if we could fix it, but without some debugging help that's going to be pretty much impossible. If you can recompile dhclient with debugging symbols and when it goes into a spin, use the debugger to figure out where it's spinning, that would be useful. See this post for details: http://lists.freebsd.org/pipermail/freebsd-current/2005-August/054224.html I'm recompiling at the moment. Will follow up with debugging results when it fails. -- Kelly D. Grills [EMAIL PROTECTED] pgpT2QVOSEkvS.pgp Description: PGP signature
dhclient wedged
I reported this problem a few OS versions back... pre-6. I'm on FreeBSD-6-STABLE, and I've found the dhclient once again wedged in a mode that was eating a lot of CPU. The solution is to kill it, and restart. I'm on Comcast's network, so I don't really know if their DHCP server is doing something that FreeBSD's stock dhclient doesn't like; however, I wonder if someone else has noticed this problem, etc. _F ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
