> But htdig continues to hang and, also as you said, it's leaving connections
> open.
The case I'm mentioning--a slow but open connection could be difficult to
fix. You can't just put an alarm on the connection since you might be
grabbing a huge file. About the only thing I can come up with is to kill
off connections where the transfer rate drops below a threshold.
More importantly, I'm not sure the code is always doing the correct TCP
handshaking on HTTP connections--there have been problems in the past and
no one has given the code a good debugging. Does anyone know of some
networking guys who could take a look?
> Are there plans to implement some way to go back and revisit just the URLs
> that were not available at the time of the initial scan?
This could probably be added, though it hasn't really been requested
before. In the 3.2 code, there's a flag for "minimal digging," i.e. only
indexing a set list of URLs, so this could potentially be used as a
workaround.
--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
