StefanRRichter commented on code in PR #24031:
URL: https://github.com/apache/flink/pull/24031#discussion_r1481189095
##########
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/restore/RocksDBIncrementalRestoreOperation.java:
##########
@@ -488,6 +495,8 @@ private void rescaleClipIngestDB(
List<ColumnFamilyHandle> tmpColumnFamilyHandles =
tmpRestoreDBInfo.columnFamilyHandles;
+ // Check if the data in all SST files referenced in the handle
is within the
+ // proclaimed key-groups range of the handle.
if (RocksDBIncrementalCheckpointUtils.isSstDataInKeyGroupRange(
Review Comment:
It is possible to import the case in your example, but the cost of detecting
the case is that we need to compare all handles against each other which we can
only do after all DBs have been opened. I'm excluding this case on purpose to
avoid it, because I also assume that this missed opportunities will be very
very rare. In particular because subtasks over time will compact their data to
the proclaimed range from normal compaction activity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
