[
https://issues.apache.org/jira/browse/FLINK-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831954#comment-15831954
]
ASF GitHub Bot commented on FLINK-5214:
---------------------------------------
GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/3183
[backport] [FLINK-5214] [FLINK-5229] Backport StreamTask and StreamOperator
checkpoint cleanup
Backport of #3179 and #3178 onto the `release-1.2` branch.
Adds exception handling to the stream operators for the snapshotState
method. A failing
snapshot operation will trigger the clean up of all so far generated state
resources.
This will avoid that in case of a failing snapshot operation resources
(e.g. files) are
left behind.
This PR adds operator state cleanup to the StreamTask class. If a stream
task contains multiple
stream operators, then every operator is checkpointed. In case that a
snapshot operation fails
all state handles and OperatorSnapshotResults belonging to previous
operators have to be freed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink backportStateCleanup
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3183.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3183
----
commit a0edef186c49520e353fe6cdc3321ef208e1bb3b
Author: Till Rohrmann <[email protected]>
Date: 2016-12-01T12:25:05Z
[FLINK-5214] Clean up checkpoint data in case of a failing checkpoint
operation
Adds exception handling to the stream operators for the snapshotState
method. A failing
snapshot operation will trigger the clean up of all so far generated state
resources.
This will avoid that in case of a failing snapshot operation resources
(e.g. files) are
left behind.
Add test case for OperatorSnapshotResult
Add StateSnapshotContextSynchronousImplTest
Add AbstractStreamOperator failing snapshot tests
commit 5eb4c2ff00a3818c53bac6c440d83bff0be8501a
Author: Till Rohrmann <[email protected]>
Date: 2017-01-20T13:28:44Z
[FLINK-5229] [state] Cleanup of operator snapshots if subsequent operator
snapshots fail
This PR adds operator state cleanup to the StreamTask class. If a stream
task contains multiple
stream operators, then every operator is checkpointed. In case that a
snapshot operation fails
all state handles and OperatorSnapshotResults belonging to previous
operators have to be freed.
Add test cases for failing checkpoint operations in StreamTask
----
> Clean up checkpoint files when failing checkpoint operation on TM
> -----------------------------------------------------------------
>
> Key: FLINK-5214
> URL: https://issues.apache.org/jira/browse/FLINK-5214
> Project: Flink
> Issue Type: Bug
> Components: TaskManager
> Affects Versions: 1.2.0, 1.1.3
> Reporter: Till Rohrmann
> Assignee: Till Rohrmann
> Fix For: 1.2.0, 1.1.4
>
>
> When the {{StreamTask#performCheckpoint}} operation fails on a
> {{TaskManager}} potentially created checkpoint files are not cleaned up. This
> should be changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
