hemantk-12 commented on code in PR #5317:
URL: https://github.com/apache/ozone/pull/5317#discussion_r1344526786
##########
hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java:
##########
@@ -799,24 +749,20 @@ private void readCompactionLogToDAG(String
currCompactionLogPath) {
}
}
- /**
- * Load existing compaction log files to the in-memory DAG.
- * This only needs to be done once during OM startup.
- */
- public void loadAllCompactionLogs() {
+ public void addEntriesFromLogFilesToDagAndCompactionLogTable() {
synchronized (this) {
- if (compactionLogDir == null) {
- throw new RuntimeException("Compaction log directory must be set " +
- "first");
- }
- reconstructionSnapshotGeneration = 0L;
+ reconstructionSnapshotCreationTime = 0L;
+ reconstructionCompactionReason = null;
try {
try (Stream<Path> pathStream = Files.list(Paths.get(compactionLogDir))
.filter(e -> e.toString().toLowerCase()
.endsWith(COMPACTION_LOG_FILE_NAME_SUFFIX))
.sorted()) {
for (Path logPath : pathStream.collect(Collectors.toList())) {
- readCompactionLogToDAG(logPath.toString());
+ readCompactionLogFile(logPath.toString());
+ // Delete the file once entries are added to compaction table
+ // so that on next restart, only compaction log table is used.
+ Files.delete(logPath);
Review Comment:
Can you please elaborate more?
I think it is OK if files are deleted. In worst case, we would lose some
compaction logs and have to fall back to full diff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
