GitHub user StefanRRichter opened a pull request:
https://github.com/apache/flink/pull/3536
[FLINK-6048] Asynchronous snapshots for heap-based OperatorStateBackend
This PR introduces asynchronous snapshots for the heap-based
`DefaultOperatorStateBackend`. Compared to the asynchronous snapshots for the
heap-based keyed state backend, this implementation is rather simple and
eagerly generates a deep in-memory copy of the state before running the
asynchronous part of the snapshot that writes to the filesystem.
Note that this PR should later sit on top of PR #3483 and piggyback on the
async-flag that was introduced.
Furthermore, we could have a followup that actually parallelizes
checkpointing the different async backends in `AsyncCheckpointRunnable`.
Previously, this was not needed because there have only been keyed state
backends o those have been the only async backends.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StefanRRichter/flink async-opstatebackend
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3536.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3536
----
commit ff0930066fd6f9a5d54c548eff73fc4f34141b6a
Author: Stefan Richter <[email protected]>
Date: 2017-03-14T13:25:57Z
[FLINK-6048] Implement async snapshots for DefaultOperatorStateBackend
commit f38fdf7524039f2d87a4594d275959d874c4a198
Author: Stefan Richter <[email protected]>
Date: 2017-03-14T15:07:06Z
Unit tests for [FLINK-6048]
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
