JingsongLi commented on code in PR #1454:
URL: https://github.com/apache/paimon-rust/pull/1454#discussion_r4656411596


##########
crates/paimon/src/table/table_commit.rs:
##########
@@ -1423,17 +1445,48 @@ impl TableCommit {
         // Once publication starts its outcome may be unknown. These files must
         // remain available even if the response is lost or a later retry 
fails.
         let base_snapshot_uuid = 
latest_snapshot.as_ref().and_then(Snapshot::uuid);
-        let publication_error = match self
+        let publication = self
             .snapshot_commit
             .commit(
                 base_snapshot_uuid,
                 &snapshot,
                 self.table.branch(),
                 &statistics,
             )
-            .await
-        {
+            .await;
+        // A file-store rejection may be a replay of a successful conditional
+        // write. Only a different snapshot can prove that cleanup is safe.
+        // REST mutations are not replayed and return a definite rejection.
+        let publication = match publication {
+            Ok(false) if automatic_outputs_published.is_some() && 
self.table.rest_env.is_none() => {

Review Comment:
   **[P1] Keep automatic compaction outputs after an ambiguous REST response**
   
   `RESTApi::commit_snapshot` currently maps a successful HTTP response without 
a boolean `success` to `Ok(false)`. This branch treats REST `false` as proof of 
rejection. If the catalog published the snapshot before returning that 
response, the retry can find the published duplicate while 
`automatic_outputs_published` and `publication_uncertain` both remain false. 
Cleanup then deletes files referenced by the published snapshot.
   
   I reproduced this through the REST catalog with two appends to an ordinary 
append table. The server durably published COMPACT snapshot 3 and returned HTTP 
204. The second writer call returned `Ok(())`, but the snapshot's only data 
file had already been removed. Please map an absent or invalid `success` to an 
ambiguous outcome, and retain this attempt's outputs when duplicate detection 
confirms that it published them.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to