Github user StefanRRichter commented on a diff in the pull request:
https://github.com/apache/flink/pull/5582#discussion_r192347037
--- Diff:
flink-state-backends/flink-statebackend-rocksdb/src/test/java/org/apache/flink/contrib/streaming/state/RocksDBStateBackendTest.java
---
@@ -547,4 +549,30 @@ public boolean accept(File file, String s) {
return true;
}
}
+
+ private static class TestRocksDBStateBackend extends
RocksDBStateBackend {
+
+ public TestRocksDBStateBackend(AbstractStateBackend
checkpointStreamBackend, boolean enableIncrementalCheckpointing) {
+ super(checkpointStreamBackend,
enableIncrementalCheckpointing);
+ }
+
+ @Override
+ public <K> AbstractKeyedStateBackend<K> createKeyedStateBackend(
+ Environment env,
+ JobID jobID,
+ String operatorIdentifier,
+ TypeSerializer<K> keySerializer,
+ int numberOfKeyGroups,
+ KeyGroupRange keyGroupRange,
+ TaskKvStateRegistry kvStateRegistry) throws IOException
{
+
+ AbstractKeyedStateBackend<K> keyedStateBackend =
super.createKeyedStateBackend(
+ env, jobID, operatorIdentifier, keySerializer,
numberOfKeyGroups, keyGroupRange, kvStateRegistry);
+
+ // We ignore the range deletions on production, but
when we are running the tests we shouldn't ignore it.
--- End diff --
Yes, I think that makes sense. My first question was if you think that
using normal deletes would be prohibitive expensive. I think there is some
threshold where restore+single deletes is cheaper than no restore+filtered
inserts. So the question is, for up to how much of the key group fraction which
strategy is the best. Do you think the single deletes are just suitable to
remove only like 1-2 key-groups or up to which point will this be better than
starting with an empty database?
---
