Stefan Groschupf wrote:
May we misunderstand each other, I do not mean tasks that crash, I mean tasks that are 20 times slower on one machine as the other tasks on the other machines.
Ah, I call that "speculative re-exectution". Nutch does not yet implement that.
I don't think speculative re-execution of tasks would help much with fetching, since a fetch task that is slow on one machine will probably be slow on another. What would probably make the fetcher faster is to use Thread.kill() on fetcher threads which have exceeded a timeout, and then replace them with a new Fetcher thread.
Speculative re-execution is among the list of features we'd like to add, but it is not a high priority for me.
Doug
