JingsongLi commented on PR #9397: URL: https://github.com/apache/paimon/pull/9397#issuecomment-5448888391
**[P1] Preserve replacement files after overwrite cleanup has committed** In `paimon-core/src/main/java/org/apache/paimon/table/format/FormatTableCommit.java:625-643`, an overwrite permanently removes the old data at lines 243-271 before `publishMessages`. However, `preservePublishedTargetOnAbort` is not set until the later metadata phase. If one parallel publish succeeds and another fails (or multipart completion succeeds but its response is lost), the catch path calls `abort`, and `deletePublishedFile(committer.targetPath())` removes the successfully or possibly published replacement as well. The old files are already gone, so the partition/table can be left with neither the old nor the replacement data. A `clean` failure after all publishes has the same outcome. Please transition overwrite messages to a state where their targets are not reclaimed after old-data cleanup succeeds and before publishing begins, while still cleaning any staging resources. Regression coverage should include partial parallel publish failure, multipart completion response loss, and post-publish `clean` failure under overwrite. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
