Hi Martin, Havve you checked that all mappers are working while parsing job is running? How many URLs are you trying to parse here?
On Friday, July 19, 2013, Martin Aesch <[email protected]> wrote: > Dear nutchers, > > Having Nutch 2.2.1/HBase 0.90.6/Hadoop 1.1.2/6Mappers/6Reducers/Core > i7-3770/32GB (no swap)/2x3TB > > When I parse (in mapper, 6 simultaneously running map-tasks), this is > very slow. Max load is ~1.5, max iowait is 5%, max CPU per task is only > 30%, max CPU for hmaster is about 30%. iotop in consequence also shows > low numbers. > > Since parsing is a CPU-intensive job and all IO-stuff is on very low > level, I wonder why parsing does not work faster und with full CPU > usage. It really takes a long time to finish. Where might be the > bottleneck? > > Thanks for any advice, > Martin > > > > > -- *Lewis*
