[
https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Attachment: patch-5785.txt
Attaching patch for review.
Patch does the following:
Reads the configuration mapred.committer.job.setup.cleanup.needed (default
value is true). If the configuration is set to false. JobInProgress.initTasks
does not initialize setup and cleanup tasks, creates _temporary directory by
calling FileOutputCommitter.setupJob(); also changes job state to Running. The
directory gets removed in garbageCollect() method.
To inform state change after initTasks, initJob method is added in
TaskTrackerManager (implemented by JobTracker). JT.initJob calls JIP.initTasks
without JobTracker lock. Once initialization completes/fails, if there is state
change. it locks JobTracker and informs listeners.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks
> for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
