[
https://issues.apache.org/jira/browse/FLINK-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995008#comment-15995008
]
ASF GitHub Bot commented on FLINK-6364:
---------------------------------------
Github user shixiaogang commented on a diff in the pull request:
https://github.com/apache/flink/pull/3801#discussion_r114565991
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java
---
@@ -769,9 +769,10 @@ public OperatorStateBackend createOperatorStateBackend(
cancelables.registerClosable(keyedStateBackend);
// restore if we have some old state
- if (null != restoreStateHandles && null !=
restoreStateHandles.getManagedKeyedState()) {
-
keyedStateBackend.restore(restoreStateHandles.getManagedKeyedState());
- }
+ Collection<KeyedStateHandle> restoreKeyedStateHandles =
+ restoreStateHandles == null ? null :
restoreStateHandles.getManagedKeyedState();
+
+ keyedStateBackend.restore(restoreKeyedStateHandles);
--- End diff --
I attempted to put the restore state in the constructor as we discussed.
But it turns out impossible.
All state backends should be registered in the task so that the backends
can be closed when the task is canceled. If we put the restoring in the
constructor of the backends, the construction of the backends may be blocked
(e.g., due to the access to HDFS). Since the construction is not completed yet,
the backend will not be registered and hence will not be closed.
> Implement incremental checkpointing in RocksDBStateBackend
> ----------------------------------------------------------
>
> Key: FLINK-6364
> URL: https://issues.apache.org/jira/browse/FLINK-6364
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: Xiaogang Shi
> Assignee: Xiaogang Shi
>
> {{RocksDBStateBackend}} is well suited for incremental checkpointing because
> RocksDB is base on LSM trees, which record updates in new sst files and all
> sst files are immutable. By only materializing those new sst files, we can
> significantly improve the performance of checkpointing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
