JingsongLi commented on code in PR #1454:
URL: https://github.com/apache/paimon-rust/pull/1454#discussion_r4656411596
##########
crates/paimon/src/table/table_commit.rs:
##########
@@ -1423,17 +1445,48 @@ impl TableCommit {
// Once publication starts its outcome may be unknown. These files must
// remain available even if the response is lost or a later retry
fails.
let base_snapshot_uuid =
latest_snapshot.as_ref().and_then(Snapshot::uuid);
- let publication_error = match self
+ let publication = self
.snapshot_commit
.commit(
base_snapshot_uuid,
&snapshot,
self.table.branch(),
&statistics,
)
- .await
- {
+ .await;
+ // A file-store rejection may be a replay of a successful conditional
+ // write. Only a different snapshot can prove that cleanup is safe.
+ // REST mutations are not replayed and return a definite rejection.
+ let publication = match publication {
+ Ok(false) if automatic_outputs_published.is_some() &&
self.table.rest_env.is_none() => {
Review Comment:
**[P1] Keep automatic compaction outputs after an ambiguous REST response**
`RESTApi::commit_snapshot` currently maps a successful HTTP response without
a boolean `success` to `Ok(false)`. This branch treats REST `false` as proof of
rejection. If the catalog published the snapshot before returning that
response, the retry can find the published duplicate while
`automatic_outputs_published` and `publication_uncertain` both remain false.
Cleanup then deletes files referenced by the published snapshot.
I reproduced this through the REST catalog with two appends to an ordinary
append table. The server durably published COMPACT snapshot 3 and returned HTTP
204. The second writer call returned `Ok(())`, but the snapshot's only data
file had already been removed. Please map an absent or invalid `success` to an
ambiguous outcome, and retain this attempt's outputs when duplicate detection
confirms that it published them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]