[ https://issues.apache.org/jira/browse/FLINK-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940820#comment-15940820 ]
ASF GitHub Bot commented on FLINK-5715: --------------------------------------- Github user StefanRRichter closed the pull request at: https://github.com/apache/flink/pull/3602 > Asynchronous snapshotting for HeapKeyedStateBackend > --------------------------------------------------- > > Key: FLINK-5715 > URL: https://issues.apache.org/jira/browse/FLINK-5715 > Project: Flink > Issue Type: New Feature > Components: State Backends, Checkpointing > Affects Versions: 1.3.0 > Reporter: Stefan Richter > Assignee: Stefan Richter > Fix For: 1.3.0 > > > Blocking snapshots render the HeapKeyedStateBackend practically unusable for > many user in productions. Their jobs can not tolerate stopped processing for > the time it takes to write gigabytes of data from memory to disk. > Asynchronous snapshots would be a solution to this problem. The challenge for > the implementation is coming up with a copy-on-write scheme for the in-memory > hash maps that build the foundation of this backend. After taking a closer > look, this problem is twofold. First, providing CoW semantics for the hashmap > itself, as a mutible structure, thereby avoiding costly locking or blocking > where possible. Second, CoW for the mutable value objects, e.g. through > cloning via serializers. -- This message was sent by Atlassian JIRA (v6.3.15#6346)