[
https://issues.apache.org/jira/browse/FLINK-7783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211016#comment-16211016
]
ASF GitHub Bot commented on FLINK-7783:
---------------------------------------
GitHub user aljoscha opened a pull request:
https://github.com/apache/flink/pull/4863
[FLINK-7783] Don't always remove checkpoints in
ZooKeeperCompletedCheckpointStore#recover()
This is a bit simpler now than described on the issue: We simply never
delete checkpoints except when they are subsumed. We always fail completely
when we try to retrieve a checkpoint handle, both in `getLatestCheckpoint()`
and `getAllCheckpoints()`. The reason for this are incremental state handles:
`getAllCheckpoints()` is used to rebuild the dependency information between the
incremental state handles when restoring. If we only return a partial list of
state handles from `getAllCheckpoints()` then this logic wouldn't work anymore,
that's why we have to fail in this method as soon as one handle is not
retrievable.
I could change `getLatestCheckpoint()` to ignore broken handles and return
an older one (in fact I have the code for this ready) but it doesn't make sense
because `getAllCheckpoints()` is also called when `getLatestCheckpoint()` is
called, meaning that restoring will fail anyways.
R: @StefanRRichter AND/OR @tillrohrmann
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aljoscha/flink
jira-7783-zookeeper-state-store-fix-simplified
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4863.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4863
----
commit bbb105a997594299ab815ec176f1893dd02ad8b9
Author: Aljoscha Krettek <[email protected]>
Date: 2017-10-19T13:06:41Z
[FLINK-7783] Don't always remove checkpoints in
ZooKeeperCompletedCheckpointStore#recover()
----
> Don't always remove checkpoints in ZooKeeperCompletedCheckpointStore#recover()
> ------------------------------------------------------------------------------
>
> Key: FLINK-7783
> URL: https://issues.apache.org/jira/browse/FLINK-7783
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Affects Versions: 1.4.0, 1.3.2
> Reporter: Aljoscha Krettek
> Assignee: Aljoscha Krettek
> Priority: Blocker
> Fix For: 1.4.0, 1.3.3
>
>
> Currently, we always delete checkpoint handles if they (or the data from the
> DFS) cannot be read:
> https://github.com/apache/flink/blob/91a4b276171afb760bfff9ccf30593e648e91dfb/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCompletedCheckpointStore.java#L180
> This can lead to problems in case the DFS is temporarily not available, i.e.
> we could inadvertently
> delete all checkpoints even though they are still valid.
> A user reported this problem on the mailing list:
> https://lists.apache.org/thread.html/9dc9b719cf8449067ad01114fedb75d1beac7b4dff171acdcc24903d@%3Cuser.flink.apache.org%3E
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
