[
https://issues.apache.org/jira/browse/NUTCH-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608049#comment-16608049
]
ASF GitHub Bot commented on NUTCH-2637:
---------------------------------------
sinsi404 opened a new pull request #381: NUTCH-2637 fix number of reducers to
run
URL: https://github.com/apache/nutch/pull/381
I'm kind a new to this, so sorry if i'm wrong.
The thing is the number of fetcher reducers are currently set to the value
of "mapreduce.job.maps" when the arg not passed. It should be
"mapreduce.job.reduces".
JIRA URL: https://issues.apache.org/jira/browse/NUTCH-2637
thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Number of fetcher reducers is misconfigured when the arg not passed
> -------------------------------------------------------------------
>
> Key: NUTCH-2637
> URL: https://issues.apache.org/jira/browse/NUTCH-2637
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 2.3, 2.3.1
> Reporter: Fumio Nakajima
> Priority: Minor
>
> I'm kind a new to this, so sorry if i'm wrong.
> The thing is the number of fetcher reducers are currently set to the value
> of "mapreduce.job.maps" when the arg not passed. It should be
> "mapreduce.job.reduces".
>
> [https://github.com/apache/nutch/blob/2.x/src/java/org/apache/nutch/fetcher/FetcherJob.java#L216]
> Line: 216, branch-2.X
> {code:java}
> if (numTasks == null || numTasks < 1) {
> currentJob.setNumReduceTasks(currentJob.getConfiguration().getInt(
> "mapreduce.job.maps", currentJob.getNumReduceTasks()));
> } else {
> currentJob.setNumReduceTasks(numTasks);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
