[
https://issues.apache.org/jira/browse/MAPREDUCE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990328#comment-12990328
]
Joydeep Sen Sarma commented on MAPREDUCE-2206:
----------------------------------------------
+1 from my side.
one thing is that very very few people will be aware that this can be turned
off. in particular - i think the default outputformats don't need a task
cleanup. i am wondering how this can be turned on automatically for more use
cases.
- we can make the setting a default one in hive-default.xml - i will file a
jira for that.
- how about hadoop streaming? can we turn task cleanup off if hadoop streaming
is used with the (default) fileoutputformat?
> The task-cleanup tasks should be optional
> -----------------------------------------
>
> Key: MAPREDUCE-2206
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2206
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.23.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2206.txt
>
>
> For job does not use OutputCommitter.abort(), this should be able to turn off.
> This improves the latency of the job because failed tasks are often the
> bottleneck of the jobs.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
