Zerolzj commented on code in PR #10582:
URL: https://github.com/apache/paimon/pull/10582#discussion_r4291905309
##########
paimon-core/src/main/java/org/apache/paimon/operation/FileStoreCommitImpl.java:
##########
@@ -725,7 +726,14 @@ public void abort(List<CommitMessage> commitMessages) {
List<DataFileMeta> dataFilesToDelete = new ArrayList<>();
dataFilesToDelete.addAll(commitMessage.newFilesIncrement().newFiles());
dataFilesToDelete.addAll(commitMessage.newFilesIncrement().changelogFiles());
-
dataFilesToDelete.addAll(commitMessage.compactIncrement().compactAfter());
+ // Level upgrades reuse files which are still referenced by the
previous snapshot.
+ Set<Path> compactInputs =
+ commitMessage.compactIncrement().compactBefore().stream()
+ .map(dataPathFactory::toPath)
+ .collect(Collectors.toSet());
+ commitMessage.compactIncrement().compactAfter().stream()
+ .filter(file ->
!compactInputs.contains(dataPathFactory.toPath(file)))
Review Comment:
Updated in .\n\nAbort cleanup now protects all physical paths from ,
including existing sidecars, and deletes only output paths outside that
set.\n\nThe regression now covers a reused main file with a newly created
lookup SST, for both internal and external paths, including repeated abort.
Both cases failed on the previous head and pass with this fix.\n\nAll 117
related tests passed locally on both JDK 8 and JDK 17, with Checkstyle and
Spotless enabled.\n\nPlease take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
