Agree. I've already implemented something like this, and also fire up new threads if the number of timed-out threads exceed a % of the total number. Unfortunately on some crawls, I ran into SocketExceptions (Too many files open)..
On Tue, 29 Mar 2005 15:24:54 -0500, andy wrote: >�Nevertheless, perhaps we should implement a timeout for >�fetcher/parser threads to protect against renegade plugins? �Maybe >�if a fetcher thread hasn't checked in after X seconds, the thread >�is interrupted.
