[
https://issues.apache.org/jira/browse/FLINK-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924466#comment-15924466
]
ASF GitHub Bot commented on FLINK-6048:
---------------------------------------
GitHub user StefanRRichter opened a pull request:
https://github.com/apache/flink/pull/3536
[FLINK-6048] Asynchronous snapshots for heap-based OperatorStateBackend
This PR introduces asynchronous snapshots for the heap-based
`DefaultOperatorStateBackend`. Compared to the asynchronous snapshots for the
heap-based keyed state backend, this implementation is rather simple and
eagerly generates a deep in-memory copy of the state before running the
asynchronous part of the snapshot that writes to the filesystem.
Note that this PR should later sit on top of PR #3483 and piggyback on the
async-flag that was introduced.
Furthermore, we could have a followup that actually parallelizes
checkpointing the different async backends in `AsyncCheckpointRunnable`.
Previously, this was not needed because there have only been keyed state
backends o those have been the only async backends.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StefanRRichter/flink async-opstatebackend
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3536.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3536
----
commit ff0930066fd6f9a5d54c548eff73fc4f34141b6a
Author: Stefan Richter <[email protected]>
Date: 2017-03-14T13:25:57Z
[FLINK-6048] Implement async snapshots for DefaultOperatorStateBackend
commit f38fdf7524039f2d87a4594d275959d874c4a198
Author: Stefan Richter <[email protected]>
Date: 2017-03-14T15:07:06Z
Unit tests for [FLINK-6048]
----
> Asynchronous snapshots for heap-based operator state backends
> -------------------------------------------------------------
>
> Key: FLINK-6048
> URL: https://issues.apache.org/jira/browse/FLINK-6048
> Project: Flink
> Issue Type: New Feature
> Components: State Backends, Checkpointing
> Affects Versions: 1.3.0
> Reporter: Stefan Richter
> Assignee: Stefan Richter
> Fix For: 1.3.0
>
>
> The synchronous checkpointing mechanism of heap-based operator state backends
> blocks element processing for the duration of the checkpoint.
> We could implement an heap-based operator state backend that allows for
> asynchronous checkpoints.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
