[
https://issues.apache.org/jira/browse/FLINK-10074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580624#comment-16580624
]
Thomas Weise edited comment on FLINK-10074 at 8/15/18 4:09 AM:
---------------------------------------------------------------
[~till.rohrmann] it is probably too difficult to retain the count in the JM
failure case, but how about making it correct for other failures? When the JM
fails - which is a very small probability - we would reset / not retain the
count. The majority of failures that we are concerned with are transient issues
in TMs where subtasks just get redeployed, much fewer cases TM machine
failures. For all these we could have a more accurate behavior by retaining the
count and failing right away on the next checkpoint failure.
was (Author: thw):
[~till.rohrmann] it is probably unreasonably hard to retain the count in the JM
failure case, but how about making it correct for other failures? When the JM
fails - which is a very small probability - we would reset / not retain the
count. The majority of failures that we are concerned with are transient issues
in TMs where subtasks just get redeployed, much fewer cases TM machine
failures. For all these we could have a more accurate behavior by retaining the
count and failing right away on the next checkpoint failure.
> Allowable number of checkpoint failures
> ----------------------------------------
>
> Key: FLINK-10074
> URL: https://issues.apache.org/jira/browse/FLINK-10074
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing
> Reporter: Thomas Weise
> Assignee: vinoyang
> Priority: Major
>
> For intermittent checkpoint failures it is desirable to have a mechanism to
> avoid restarts. If, for example, a transient S3 error prevents checkpoint
> completion, the next checkpoint may very well succeed. The user may wish to
> not incur the expense of restart under such scenario and this could be
> expressed with a failure threshold (number of subsequent checkpoint
> failures), possibly combined with a list of exceptions to tolerate.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
