[
https://issues.apache.org/jira/browse/FLINK-10074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580624#comment-16580624
]
Thomas Weise edited comment on FLINK-10074 at 8/15/18 4:07 AM:
---------------------------------------------------------------
[~till.rohrmann] it is probably unreasonably hard to retain the count in the JM
failure case, but how about making it correct for other failures? When the JM
fails - which is a very small probability - we would reset / not retain the
count. The majority of failures that we are concerned with are transient issues
in TMs where subtasks just get redeployed, much fewer cases TM machine
failures. For all these we could have a more accurate behavior by retaining the
count and failing right away on the next checkpoint failure.
was (Author: thw):
[~till.rohrmann] it is probably unreasonably hard to do for the JM failure
case, but how about making it correct with best effort? Only when the JM fails,
we would not retain the count, which is a very small probability. The majority
of failures are transient issues in TMs where subtasks just get redeployed,
much fewer cases TM machine failures. For all these we could have a more
accurate behavior by retaining the count and failing right away on the next
checkpoint failure.
> Allowable number of checkpoint failures
> ----------------------------------------
>
> Key: FLINK-10074
> URL: https://issues.apache.org/jira/browse/FLINK-10074
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing
> Reporter: Thomas Weise
> Assignee: vinoyang
> Priority: Major
>
> For intermittent checkpoint failures it is desirable to have a mechanism to
> avoid restarts. If, for example, a transient S3 error prevents checkpoint
> completion, the next checkpoint may very well succeed. The user may wish to
> not incur the expense of restart under such scenario and this could be
> expressed with a failure threshold (number of subsequent checkpoint
> failures), possibly combined with a list of exceptions to tolerate.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
