JingsongLi commented on code in PR #852:
URL: https://github.com/apache/incubator-paimon/pull/852#discussion_r1163513217
##########
paimon-core/src/main/java/org/apache/paimon/operation/FileStoreExpireImpl.java:
##########
@@ -327,4 +351,62 @@ private void writeEarliestHint(long earliest) {
throw new RuntimeException(e);
}
}
+
+ private void tryDeleteDirectories(Map<BinaryRow, Set<Integer>>
changedBuckets) {
+ // All directory paths are deduplicated and sorted by hierarchy level
+ Map<Integer, Set<Path>> deduplicate = new HashMap<>();
+ // help to deduplicate
+ Set<Path> uncheck = new HashSet<>();
+ for (Map.Entry<BinaryRow, Set<Integer>> entry :
changedBuckets.entrySet()) {
+ // try to delete bucket directories
+ boolean allAttemptsSuccessful = true;
+ for (Integer bucket : entry.getValue()) {
+ allAttemptsSuccessful =
+ allAttemptsSuccessful
+ && tryDeleteEmptyDirectory(
+ pathFactory.bucketPath(entry.getKey(),
bucket));
+ }
+
+ List<Path> hierarchicalPaths =
pathFactory.getHierarchicalPartitionPath(entry.getKey());
+ int hierarchies = hierarchicalPaths.size();
+ if (hierarchies == 0) {
+ continue;
+ }
+
+ if (allAttemptsSuccessful
+ &&
tryDeleteEmptyDirectory(hierarchicalPaths.get(hierarchies - 1))) {
+ // deduplicate high level partition directories
+ for (int hierarchy = 0; hierarchy < hierarchies - 1;
hierarchy++) {
+ Path path = hierarchicalPaths.get(hierarchy);
+ if (!uncheck.contains(path)) {
+ deduplicate.computeIfAbsent(hierarchy, i -> new
HashSet<>()).add(path);
+ }
+ }
+ } else {
+ // if there has left bucket directories or the deepest
partition directory, give up
+ // to check this partition and add high level paths to uncheck
list
+ if (hierarchies > 1) {
+ uncheck.addAll(hierarchicalPaths.subList(1, hierarchies));
Review Comment:
`hierarchicalPaths.subList(1, hierarchies)` 0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
