[
https://issues.apache.org/jira/browse/MAPREDUCE-7300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved MAPREDUCE-7300.
---------------------------------------
Resolution: Won't Fix
mapreduce-7341 is intended to replace v2 algorithm on object stores; it will
provide safe FS commit semantics and higher job commit stats. Won't be as fast
as V2 as job commit now does work, but it will be a lot faster, especially on
deep directory trees
> PathOutputCommitter to add method failedTaskAttemptCommitRecoverable()
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-7300
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7300
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
>
> Related to MAPREDUCE-7282 a variant solution
> # we add a new method for committers to declare whether they can recover from
> a task attempt commit failure
> # default = true; v2 and (external) EMR spark committers can return false
> execution engine - MR, Spark, can look at this after a task attempt fails to
> commit and decide what to do
> recoverable: execute/commit another task attempt
> non-recoverable, one of (Configured)
> * warn and continue
> * abort the job
> with the job abort option, users would be confident that if a failure
> happened during the commit phase, they'd know about it and choose how to
> recover.
> I'd use a long/unusual name, so that in, say, Spark, reflection to could be
> used to find and call the method & so compile against older releases
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
