Hello Suraj, You can safely increase the number of reducers for UpdateHostDB to as high as you like.
Regards, Markus -----Original message----- > From:Suraj Singh <ssi...@olbico.nl> > Sent: Monday 18th March 2019 11:41 > To: email@example.com > Subject: Increasing the number of reducer in UpdateHostDB > > Hi All, > > Can I increase the number of reducer in UpdateHostDB step? Currently it is > running with 1 reducer. > Will it impact the crawling in any way? > > Current command in crawl script: > __bin_nutch updatehostdb -crawldb "$CRAWL_PATH"/crawldb -hostdb > "$CRAWL_PATH"/hostdb > > Can I update it to: > __bin_nutch updatehostdb -D mapreduce.job.reduces=32 -crawldb > "$CRAWL_PATH"/crawldb -hostdb "$CRAWL_PATH"/hostdb > > Thanks it advance. > > Regards, > Suraj Singh > >