Thomas Reinke <[EMAIL PROTECTED]> writes: > We've noted in a few cases that wget can hang on connect() due to a > lack of any form of timeout management. We've made a change to the > routine "connect_to_one" in "connect.c" that will implement a > timeout mechanism on connect without the use of signals or alarms.
Please note that the preferred way of distributing changes to source code is in the form of a patch. Take a look at the file `PATCHES' in the Wget distribution, which explains how to produce one. > I've attached the modified version of "connect.c", (post configure), > as well as the specific changes that were made below. We've found > this appears to be a generic mechanism that works extremely for > pretty much any type of situation we've encountered (used it to > essentially crawl the entire web, never locking up once.) > > This specific version tested on Redhat 6.2, kernel 2.2.19-6.2.7 Thanks for the contribution. Shortening connect() is probably a useful feature in some circumstances. But one problem with this implementation is portability -- I'm pretty sure that some systems don't support FIONBIO. In fact, Wget should work even on systems without a select() call. Several days ago I've taken a look at how libcurl handles non-blocking sockets, and I was appalled at the complexity of the configure checks required to support a large variety of systems. Implementing the same feature in Wget should probably require more investigation and a lot of testing on different platforms. So far I've been consistently reaching the conclusion that connect timeouts are simply not worth the effort. P.S. You mean you've crawled the entire web with Wget? Cool.
