[
https://issues.apache.org/jira/browse/FLINK-6034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940129#comment-15940129
]
ASF GitHub Bot commented on FLINK-6034:
---------------------------------------
Github user StefanRRichter commented on a diff in the pull request:
https://github.com/apache/flink/pull/3531#discussion_r107874353
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
@@ -761,6 +769,13 @@ private void restoreKVStateMetaData() throws
IOException, ClassNotFoundException
private void restoreKVStateData() throws IOException,
RocksDBException {
//for all key-groups in the current state handle...
for (Tuple2<Integer, Long> keyGroupOffset :
currentKeyGroupsStateHandle.getGroupRangeOffsets()) {
+ int keyGroup = keyGroupOffset.f0;
+
+ // Skip those key groups that do not belong to
the backend
+ if
(!rocksDBKeyedStateBackend.getKeyGroupRange().contains(keyGroup)) {
--- End diff --
The same comment I had on the `HeapKeyedStateBackend` also applies here: I
think the post-filter is no longer required after the change i suggested for
`StateAssignmentOperation`.
> Add KeyedStateHandle for the snapshots in keyed streams
> -------------------------------------------------------
>
> Key: FLINK-6034
> URL: https://issues.apache.org/jira/browse/FLINK-6034
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: Xiaogang Shi
> Assignee: Xiaogang Shi
>
> Currently, the only type of the snapshots in keyed streams is
> {{KeyGroupsStateHandle}} which is full and store the states one group after
> another. With the introduction of incremental checkpointing, we need a higher
> level abstraction of keyed snapshots to allow flexible snapshot formats.
> The implementation of {{KeyedStateHandle}} s may vary a lot in different
> backends. The only information needed in {{KeyedStateHandle}} s is their key
> group range. When recovering the job with a different degree of parallelism,
> {{KeyedStateHandle}} s will be assigned to those subtasks whose key group
> ranges overlap with their ranges.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
