Thank you Markus.
-Original Message-
From: Markus Jelsma
Sent: Monday, 18 March 2019 11:49
To: user@nutch.apache.org
Subject: RE: Increasing the number of reducer in UpdateHostDB
Hello Suraj,
You can safely increase the number of reducers for UpdateHostDB to as high as
you like.
Regards,
Markus
-Original message-
> From:Suraj Singh
> Sent: Monday 18th March 2019 11:41
> To: user@nutch.apache.org
> Subject: Increasing the number of reducer in UpdateHostDB
>
> Hi All,
>
> Can I increase the number of reducer in UpdateHostDB step? Currently it is
> running with 1 reducer.
> Will it impact the crawling in any way?
>
> Current command in crawl script:
> __bin_nutch updatehostdb -crawldb "$CRAWL_PATH"/crawldb -hostdb
> "$CRAWL_PATH"/hostdb
>
> Can I update it to:
> __bin_nutch updatehostdb -D mapreduce.job.reduces=32 -crawldb
> "$CRAWL_PATH"/crawldb -hostdb "$CRAWL_PATH"/hostdb
>
> Thanks it advance.
>
> Regards,
> Suraj Singh
>
>
