Hi, 1. I have disabled speculative tasks by setting it to false in hadoop-site.xml
2. Now I notice that the fetcher does not complete the whole fetchlist. 3. By adding additional logging info in generate I see 20000 links being generated, but the fetcher without any indication to any error just fetch some 200-1000 urls. 4. Hadoop loging just writes task ... has completed. 5. I am using latest Hadoop version. Gal.
