Steve Loughran created MAPREDUCE-7300:
-----------------------------------------
Summary: PathOutputCommitter to add method
failedTaskAttemptCommitRecoverable()
Key: MAPREDUCE-7300
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7300
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 3.3.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Related to HADOOP-7282, a variant solution
# we add a new method for committers to declare whether they can recover from a
task attempt commit failure
# default = true; v2 and (external) EMR spark committers can return false
execution engine - MR, Spark, can look at this after a task attempt fails to
commit and decide what to do
recoverable: execute/commit another task attempt
non-recoverable, one of (Configured)
* warn and continue
* abort the job
with the job abort option, users would be confident that if a failure happened
during the commit phase, they'd know about it and choose how to recover.
I'd use a long/unusual name, so that in, say, Spark, reflection to could be
used to find and call the method & so compile against older releases
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
