[ https://issues.apache.org/jira/browse/MAPREDUCE-434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated MAPREDUCE-434: ------------------------------------ Attachment: MAPREDUCE-434.2.patch Submitting new patch sync'd with trunk. When testing this patch, I uncovered a problematic interaction between mapper/reducer tasks and the {{Merger}}. Specifically, {{MergeQueue}} uses a static field {{Merger.includeFinalMerge}} which is set to true in {{MapTask}}, but is never cleared. Now that both task types can occur in the same JVM, and a {{Merger}} may be used in both (local reduce tasks did not actually instantiate a {{Merger}} in the old codepath), this flag must be made a per-{{MergeQueue}} setting. I removed the static field and pushed it into {{MergeQueue}} as a member field; the {{Merger.merge()}} methods used as constructors for {{MergeQueue}} now take a {{TaskType}} parameter which can be set to {{TaskType.MAP}} to achieve the same effect. > local map-reduce job limited to single reducer > ---------------------------------------------- > > Key: MAPREDUCE-434 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-434 > Project: Hadoop Map/Reduce > Issue Type: Bug > Environment: local job tracker > Reporter: Yoram Arnon > Assignee: Aaron Kimball > Priority: Minor > Attachments: MAPREDUCE-434.2.patch, MAPREDUCE-434.patch > > > when mapred.job.tracker is set to 'local', my setNumReduceTasks call is > ignored, and the number of reduce tasks is set at 1. > This prevents me from locally debugging my partition function, which tries to > partition based on the number of reduce tasks. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.