[
https://issues.apache.org/jira/browse/HADOOP-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711314#action_12711314
]
Vinod K V commented on HADOOP-5850:
-----------------------------------
Forgot to summarize. With this patch,
- Jobs with zero maps will still run job setup and cleanup tasks.
- If number of reduces is non-zero, reduces are run leaving behind the
corresponding number of empty part-files in the output directory.
- If the number of reduces is also zero, an empty output directory is left
behind.
- The map progress(and reduce progress if number of reduces is zero) is set to
1.0 once the job cleanup task finishes.
> map/reduce doesn't run jobs with 0 maps
> ---------------------------------------
>
> Key: HADOOP-5850
> URL: https://issues.apache.org/jira/browse/HADOOP-5850
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.20.0
> Reporter: Owen O'Malley
> Assignee: Vinod K V
> Priority: Critical
> Fix For: 0.20.1
>
> Attachments: HADOOP-5850-20090519.1.txt, HADOOP-5850-20090519.txt,
> HADOOP-5850-20090520-svn.1.txt
>
>
> Currently, the framework ignores jobs that have 0 maps. This is incorrect.
> Many pipelines need the job to run (if nothing else, to create the output
> directory!) so that subsequent jobs don't fail. Effectively, there will be no
> map tasks and the reduce tasks should immediately set up the Reducer and
> RecordWriter and then call close on both since there are no inputs to the
> reduce. I believe it should just work if we remove the check...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
