[
https://issues.apache.org/jira/browse/HADOOP-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod K V updated HADOOP-5850:
------------------------------
Attachment: HADOOP-5850-20090519.txt
Attaching patch to fix this issue.
- With this patch, jobs will 0 maps or no input still run JobSetUp, any number
of reduces( which do nothing), and the JobCleanUp task.
- Removed the 0-splits check in JobInProgress.initTasks() and added checks so
that cleanup task doesn't launch before setup tasks when number of splits is
zero.
- Renamed TestEmptyJobWithDFS to TestEmptyJob, removed HDFS dependence to
quicken the test, added checks for verifying the number of map and reduce tasks
run for an empty-job.
> map/reduce doesn't run jobs with 0 maps
> ---------------------------------------
>
> Key: HADOOP-5850
> URL: https://issues.apache.org/jira/browse/HADOOP-5850
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.20.0
> Reporter: Owen O'Malley
> Assignee: Vinod K V
> Priority: Critical
> Fix For: 0.20.1
>
> Attachments: HADOOP-5850-20090519.txt
>
>
> Currently, the framework ignores jobs that have 0 maps. This is incorrect.
> Many pipelines need the job to run (if nothing else, to create the output
> directory!) so that subsequent jobs don't fail. Effectively, there will be no
> map tasks and the reduce tasks should immediately set up the Reducer and
> RecordWriter and then call close on both since there are no inputs to the
> reduce. I believe it should just work if we remove the check...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
