wwj6591812 commented on code in PR #4545:
URL: https://github.com/apache/paimon/pull/4545#discussion_r1855756447
##########
paimon-core/src/main/java/org/apache/paimon/operation/LocalOrphanFilesClean.java:
##########
@@ -87,16 +90,27 @@ public LocalOrphanFilesClean(
table.coreOptions().deleteFileThreadNum(),
"ORPHAN_FILES_CLEAN");
}
- public List<Path> clean() throws IOException, ExecutionException,
InterruptedException {
+ public CleanOrphanFilesResult clean()
+ throws IOException, ExecutionException, InterruptedException {
List<String> branches = validBranches();
// specially handle to clear snapshot dir
- cleanSnapshotDir(branches, deleteFiles::add);
+ cleanSnapshotDir(
+ branches,
+ deleteFiles::add,
+ p -> {
+ try {
+
deletedFilesSizeInBytes.addAndGet(fileIO.getFileSize(p));
Review Comment:
> getFileSize should always be together with delete, otherwise, I feel there
may be issues with execution efficiency, getFileSize should be executed
parallelismly too.
@JingsongLi Very thanks for your suggestions.
I would like to confirm with you the code improvement plan:
Are you means that getFileStatus execute only once in
OrphanFilesClean#createFileCleaner, and fileCleaner can direct return the size
of deleted file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
