Now I'm seeing mapred running extremely slow during geneator step, at the end parsesegment step, and at updatedb step. I'm running nutch with 3000 domains in regex-urlfilter.xml on single machine. For example, in generating segment, it prints this log message forever: 2010-07-31 10:13:12,138 INFO mapred.LocalJobRunner - file:/prod/disco/data/cr1700/crawl/crawldb/current/part-00000/data:0+35468244
This could be just a configuration issue. Any idea for what might be wrong? thanks, -aj
