[
https://issues.apache.org/jira/browse/FLINK-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831850#comment-15831850
]
ASF GitHub Bot commented on FLINK-5229:
---------------------------------------
GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/3179
[FLINK-5229] [state] Cleanup of operator snapshots if subsequent operator
snapshots fail
This PR is based on #3178.
This PR adds operator state cleanup to the StreamTask class. If a stream
task contains multiple
stream operators, then every operator is checkpointed. In case that a
snapshot operation fails
all state handles and OperatorSnapshotResults belonging to previous
operators have to be freed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink streamTaskCheckpointCleanup
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3179.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3179
----
commit c1a597320dbe3c7f4514297d5ad7f2b8f416e287
Author: Till Rohrmann <[email protected]>
Date: 2016-12-01T12:25:05Z
[FLINK-5214] Clean up checkpoint data in case of a failing checkpoint
operation
Adds exception handling to the stream operators for the snapshotState
method. A failing
snapshot operation will trigger the clean up of all so far generated state
resources.
This will avoid that in case of a failing snapshot operation resources
(e.g. files) are
left behind.
Add test case for OperatorSnapshotResult
Add StateSnapshotContextSynchronousImplTest
Add AbstractStreamOperator failing snapshot tests
commit b407e812ccb9c727ee0df8d130b54a50abee5b06
Author: Till Rohrmann <[email protected]>
Date: 2017-01-20T13:28:44Z
[FLINK-5229] [state] Cleanup of operator snapshots if subsequent operator
snapshots fail
This PR adds operator state cleanup to the StreamTask class. If a stream
task contains multiple
stream operators, then every operator is checkpointed. In case that a
snapshot operation fails
all state handles and OperatorSnapshotResults belonging to previous
operators have to be freed.
Add test cases for failing checkpoint operations in StreamTask
----
> Cleanup StreamTaskStates if a checkpoint operation of a subsequent operator
> fails
> ----------------------------------------------------------------------------------
>
> Key: FLINK-5229
> URL: https://issues.apache.org/jira/browse/FLINK-5229
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing, TaskManager
> Affects Versions: 1.2.0, 1.1.3
> Reporter: Till Rohrmann
> Assignee: Till Rohrmann
> Fix For: 1.2.0, 1.1.4
>
>
> Due to chaining, a {{StreamTask}} needs to checkpoint multiple operators. If
> the first operators succeed in creating a checkpoint but a subsequent
> operator in the chain fails, the {{StreamTask}} has to clean up the already
> completed checkpoints. Otherwise we might end up with orphaned state data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
