[
https://issues.apache.org/jira/browse/YARN-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13646188#comment-13646188
]
Bikas Saha commented on YARN-614:
---------------------------------
Agree about a method that encapsulates if an rmappattempt failed with an error
we want to ignore. Perhaps rename it to countFailureToAttemptLimit() or
something like that. We can then add hysteresis logic later on for perpetual
apps for which we want to count failures only in the last hour say.
I am afraid allowing appattempt.size to exceed maxAttempts might break code
somewhere else that did not expect this to happen. Need to check thoroughly for
this.
The recovery code wont work since right now, the RM does not recover attempts
where appattempts.size() > maxattempts. eg. of above case. Look at
RMAppManager.recover(). One solution could be to move the check from
finishAttempt() to createAttempt(). finishAttempt() always enqueues a new
attempt. the new attempt creation checks if one can still be created based on
failed count etc. Another solution could be to make the RMApp go from NEW to
FAILED in the recover transition based on failed counts etc.
Having said that, recovery wont work because the mastercontainer is saved
before launching the attempt and as such does not have the exit status
populated in it. We could leave recovery for a different jira and focus on the
regular code path in this one perhaps.
> Retry attempts automatically for hardware failures or YARN issues and set
> default app retries to 1
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-614
> URL: https://issues.apache.org/jira/browse/YARN-614
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Bikas Saha
> Attachments: YARN-614-0.patch
>
>
> Attempts can fail due to a large number of user errors and they should not be
> retried unnecessarily. The only reason YARN should retry an attempt is when
> the hardware fails or YARN has an error. NM failing, lost NM and NM disk
> errors are the hardware errors that come to mind.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
