smengcl commented on code in PR #3786:
URL: https://github.com/apache/ozone/pull/3786#discussion_r992941868
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java:
##########
@@ -695,17 +877,45 @@ public synchronized void printMutableGraph(
}
}
+ private String getCompactionLogFilenameGivenCpPath(String cpPath) {
+ return cpPath + "-" + CURRENT_COMPACTION_LOG_FILENAME;
+ }
public void createSnapshot(RocksDB rocksDB) throws InterruptedException {
LOG.warn("Current time is::" + System.currentTimeMillis());
long t1 = System.currentTimeMillis();
cpPath = cpPath + lastSnapshotCounter;
+ // Delete the checkpoint dir if it already exists
+ File dir = new File(cpPath);
+ if (dir.exists()) {
+ deleteDirectory(dir); // TODO: FOR EASE OF TESTING ONLY. DO NOT DELETE
DIR WHEN MERGING
+ }
+
createCheckPoint(rocksDbPath, cpPath, rocksDB);
allSnapshots[lastSnapshotCounter] = new Snapshot(cpPath,
lastSnapshotPrefix, lastSnapshotCounter);
+ // Rename compactions.log
+ final String existingFilename = CURRENT_COMPACTION_LOG_FILENAME;
+ final String newFilename = getCompactionLogFilenameGivenCpPath(cpPath);
+ try {
+
+ // Dev: Remove target is already exists
+ if (new File(newFilename).exists()) {
+ new File(newFilename).delete();
+ }
+
+ if (new File(existingFilename).exists()) {
+ Files.move(Paths.get(existingFilename), Paths.get(newFilename));
Review Comment:
Hi George, we realized we actually couldn't `pauseBackgroundWork()` because
RDB checkpointing also counts as a kind of background work. After
`pauseBackgroundWork()` is called, the RDB checkpointing call would be stuck
indefinitely (because ALL background work is paused, and there doesn't seem to
have an option to pause only compaction -- which we could improve this in RDB,
but that would be a much longer turnaround to implement and wait for another
RDB JNI release). So we switched to using a new compaction log file name every
time a new snapshot is created in #3824 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
