[
https://issues.apache.org/jira/browse/MAPREDUCE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17605648#comment-17605648
]
Daniel Ma edited comment on MAPREDUCE-7351 at 9/17/22 1:57 AM:
---------------------------------------------------------------
ShutDownhook calback function will be triggerred once the AM process is down no
matter the corresponding job is finished or killed.
Many upper applications like Sqoop will delete the output directory on hdfs in
abortJob().
This MR will clear all the output data of a MR job including normal scenario.
With the patch logic added, the output directory will be deleted by all means
no matter the job is succeed, failed, or killed.
it should be reverted.
was (Author: daniel ma):
This MR will clear all the output data of a MR job including normal scenario.
Many upper applications like Sqoop will delete the output directory on hdfs in
abortJob().
With the patch logic added, the output directory will be deleted by all means
no matter the job is succeed, failed, or killed.
it should be reverted.
> CleanupJob during handle of SIGTERM signal
> ------------------------------------------
>
> Key: MAPREDUCE-7351
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7351
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 3.3.0
> Reporter: Prabhu Joseph
> Assignee: Shubham Gupta
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: MAPREDUCE-7351-001.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently MR CleanupJob happens when the job is either successful or fail.
> But during kill, it is not handled. This leaves all the temporary folders
> under the output path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
