[
https://issues.apache.org/jira/browse/HADOOP-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Chansler updated HADOOP-3150:
------------------------------------
Release Note:
Moved task file promotion from to the Task. When the task has finished, it will
do a commit and is declared SUCCEDED. Job cleanup is done by a separate task.
Job is declared SUCCEDED/FAILED after the cleanup task has finished. Added
public classes org.apache.hadoop.mapred.JobContext, TaskAttemptContext,
OutputCommitter and FileOutputCommiitter. Added public APIs: public
OutputCommitter getOutputCommitter() and
public void setOutputCommitter(Class<? extends OutputCommitter> theClass) in
org.apache.hadoop.mapred.JobConf
was:
1. Moved the task file promotion from the JobTracker to the Task and moved it
down into the OutputCommitter. Now, once the task has finished executing its
map/reduce code, it will do a commit if required. Then the task is declared
SUCCEDED.
2. Job cleanup is done by a separate task when all maps and reduces are
completed or if the job is killed. Job is declared SUCCEDED/FAILED after the
cleanup task has finished.
3. Added public classes org.apache.hadoop.mapred.JobContext,
org.apache.hadoop.mapred.TaskAttemptContext,
org.apache.hadoop.mapred.OutputCommitter and
org.apache.hadoop.mapred.FileOutputCommiitter
4. Added public apis: public OutputCommitter getOutputCommitter() and
public void setOutputCommitter(Class<? extends OutputCommitter> theClass) in
org.apache.hadoop.mapred.JobConf
> Move task file promotion into the task
> --------------------------------------
>
> Key: HADOOP-3150
> URL: https://issues.apache.org/jira/browse/HADOOP-3150
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.19.0
>
> Attachments: 3150 test plan.txt, 3150.patch, patch-3150.txt,
> patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
> patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
> patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
> patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
> patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
> patch-3150.txt, patch-3150.txt
>
>
> We need to move the task file promotion from the JobTracker to the Task and
> move it down into the output format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
