jojochuang commented on code in PR #9268:
URL: https://github.com/apache/ozone/pull/9268#discussion_r2535669836
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java:
##########
@@ -832,22 +779,29 @@ private String getSSTFullPath(String
sstFilenameWithoutExtension,
* "/path/to/sstBackupDir/000060.sst"]
*/
public synchronized Optional<List<String>>
getSSTDiffListWithFullPath(DifferSnapshotInfo src,
- DifferSnapshotInfo dest, Set<String> tablesToLookup,
- String sstFilesDirForSnapDiffJob) {
+ DifferSnapshotInfo dest, Map<Integer, Integer> versionMap,
TablePrefixInfo prefixInfo,
+ Set<String> tablesToLookup, String sstFilesDirForSnapDiffJob) throws
IOException {
+ int srcVersion = src.getMaxVersion();
+ if (!versionMap.containsKey(srcVersion)) {
+ throw new IOException("No corresponding dest version corresponding
srcVersion : " + srcVersion + " in " +
+ "versionMap : " + versionMap);
+ }
+ int destVersion = versionMap.get(srcVersion);
+ DifferSnapshotVersion srcSnapshotVersion = new DifferSnapshotVersion(src,
src.getMaxVersion(), tablesToLookup);
+ DifferSnapshotVersion destSnapshotVersion = new
DifferSnapshotVersion(dest, destVersion, tablesToLookup);
- Optional<List<String>> sstDiffList = getSSTDiffList(src, dest,
tablesToLookup);
+ Optional<List<SstFileInfo>> sstDiffList =
getSSTDiffList(srcSnapshotVersion, destSnapshotVersion, prefixInfo,
+ tablesToLookup, srcVersion == 0);
Review Comment:
```suggestion
// if the source snapshot is the first version (srcVersion == 0), uses
the compaction DAG path;
// otherwise, uses the fallback set-comparison path.
Optional<List<SstFileInfo>> sstDiffList =
getSSTDiffList(srcSnapshotVersion, destSnapshotVersion, prefixInfo,
tablesToLookup, srcVersion == 0);
```
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDiffUtils.java:
##########
@@ -54,40 +48,32 @@ public static boolean isKeyWithPrefixPresent(String
prefixForColumnFamily,
&& prefixForColumnFamily.compareTo(endKeyPrefix) <= 0;
}
- public static void filterRelevantSstFiles(Set<String> inputFiles,
- TablePrefixInfo tablePrefixInfo,
- Set<String> columnFamiliesToLookup,
- ManagedRocksDB... dbs) {
- filterRelevantSstFiles(inputFiles, tablePrefixInfo,
Collections.emptyMap(), columnFamiliesToLookup, dbs);
+ /**
+ * Filter sst files based on prefixes.
+ */
+ public static <T> Map<T, SstFileInfo> filterRelevantSstFiles(Map<T,
SstFileInfo> inputFiles,
Review Comment:
note: like the previous method, the parameter inputFiles is not just an
input: the map can be mutated.
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java:
##########
@@ -859,57 +813,114 @@ public synchronized Optional<List<String>>
getSSTDiffListWithFullPath(DifferSnap
*
* @param src source snapshot
* @param dest destination snapshot
+ * @param prefixInfo TablePrefixInfo to filter irrelevant SST files; can be
null.
* @param tablesToLookup tablesToLookup Set of column-family (table) names
to include when reading SST files;
* must be non-null.
* @return A list of SST files without extension. e.g. ["000050", "000060"]
Review Comment:
```suggestion
* @param useCompactionDag whether to consult the compaction DAG to
compute the SST diff.
* If true, the method uses compaction history to
produce an
* incremental diff; if false, a full (conservative)
diff will be returned.
* @return A list of SST files without extension. e.g. ["000050", "000060"]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
