[
https://issues.apache.org/jira/browse/TEZ-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835691#comment-16835691
]
Jonathan Eagles commented on TEZ-2249:
--------------------------------------
Looked at MapReduce for a similar feature, but there is none. It is susceptible
to the same race condition. I have seen this occur recently and the outcome can
be bad since temporary directory (and presumably files) can show up after the
vertex stage commits. If subsequent stages are triggered based on a SUCCESS
file being written, this can cause issues and contents change after the SUCCESS
marker is created (a '_SUCCESS' file).
If there is still interest, I could help work on this patch (giving [~zjffdu]
proper credit) as assignee isn't able to work on this.
> Wait for all task attempt finished before moving Task to finished state
> -----------------------------------------------------------------------
>
> Key: TEZ-2249
> URL: https://issues.apache.org/jira/browse/TEZ-2249
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Priority: Major
> Attachments: TEZ-2249-1.patch
>
>
> 2 cases:
> * If Task needs to move the SUCCEEDED, then committing may happens while
> there's still task attempt running.
> * If Tasks needs to move to FAILED/KILLED/ERROD, then aborting may happens
> while there's still task attempt running.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
