Github user StefanRRichter commented on a diff in the pull request:
https://github.com/apache/flink/pull/3870#discussion_r116175297
--- Diff:
flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBIncrementalKeyedStateHandle.java
---
@@ -180,69 +176,66 @@ public long getStateSize() {
@Override
public void registerSharedStates(SharedStateRegistry stateRegistry) {
+
Preconditions.checkState(!registered, "The state handle has
already registered its shared states.");
- for (Map.Entry<String, StreamStateHandle> newSstFileEntry :
newSstFiles.entrySet()) {
- SstFileStateHandle stateHandle = new
SstFileStateHandle(newSstFileEntry.getKey(), newSstFileEntry.getValue());
+ for (Map.Entry<String, StreamStateHandle> newSstFileEntry :
unregisteredSstFiles.entrySet()) {
--- End diff --
For FLINK-6545 we need to familiarize the savepoint serializer with the new
incremental handles, but currently they are in the RocksDB package, invisible
for savepoint classes.
I am currently already working on making incremental snapshots a concept on
a higher abstraction level, that is less tightly coupled to RocksDB. I think
that we can then have incremental snapshots also for other backends, e.g. the
memory based. The abstraction is simply, that all incremental snapshots can be
divided into backend meta data, private data, newly created shared data and
referenced shared data. Also the placeholder handle will become publicly
available. Would this address comments? I could make those preparations already
part of this PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
