hemantk-12 commented on code in PR #5035:
URL: https://github.com/apache/ozone/pull/5035#discussion_r1258574640
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RocksDatabase.java:
##########
@@ -979,9 +982,17 @@ public void deleteFilesNotMatchingPrefix(
+ " {} from db: {}", sstFileName,
liveFileMetaData.columnFamilyName(), db.get().getName());
db.get().deleteFile(sstFileName);
+ filesToBeDeleted.add(new File(liveFileMetaData.path(),
+ liveFileMetaData.fileName()));
}
}
}
+
+ Iterator<File> files = filesToBeDeleted.iterator();
+ while (files.hasNext()) {
Review Comment:
Why do we need to wait for file deletion? Once manifest is updated, it
doesn't matter if file exist or got deleted.
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java:
##########
@@ -3577,18 +3578,24 @@ public List<OzoneAcl> getAcl(OzoneObj obj) throws
IOException {
* @return If checkpoint is installed successfully, return the
* corresponding termIndex. Otherwise, return null.
*/
- public TermIndex installSnapshotFromLeader(String leaderId) {
+ public synchronized TermIndex installSnapshotFromLeader(String leaderId) {
if (omRatisSnapshotProvider == null) {
LOG.error("OM Snapshot Provider is not configured as there are no peer "
+
"nodes.");
return null;
}
+ java.util.Optional<SstFilteringService> sstFilteringService =
+ java.util.Optional.ofNullable(
+ keyManager.getSnapshotSstFilteringService());
DBCheckpoint omDBCheckpoint;
+ sstFilteringService.ifPresent(BackgroundService::shutdown);
Review Comment:
I don't think it is a good idea to shutdown and start the service like this.
I think we should use lock as it is done in other places like following:
*
https://github.com/apache/ozone/blob/0862a76ef178f427a272c335374760c87c885119/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java#L1167
*
https://github.com/apache/ozone/blob/1ae0401b1ac662c021b37b68ef46f4b9a0909890/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/DBCheckpointServlet.java#L174
@GeorgeJahad can add or correct me if we can't use that.
I see we have a lock:
https://github.com/apache/ozone/blob/e950ecee512fd87a9c20c233ba6ed224acd2c6e4/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/SstFilteringService.java#L164
We may need to place this lock before starting the SST file cleanup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
