JingsongLi commented on code in PR #1417:
URL: https://github.com/apache/paimon-rust/pull/1417#discussion_r4601350292
##########
crates/paimon/src/table/table_commit.rs:
##########
@@ -1423,16 +1423,37 @@ impl TableCommit {
// Once publication starts its outcome may be unknown. These files must
// remain available even if the response is lost or a later retry
fails.
let base_snapshot_uuid =
latest_snapshot.as_ref().and_then(Snapshot::uuid);
- let publication_error = match self
+ let publication = self
.snapshot_commit
.commit(
base_snapshot_uuid,
&snapshot,
self.table.branch(),
&statistics,
)
- .await
- {
+ .await;
+ // Storage may replay a successful conditional write after losing its
+ // response. Establish ownership before treating false as a rejection.
+ let publication = match publication {
+ Ok(false) => match
self.snapshot_manager.get_snapshot_uncached(snapshot.id()).await {
Review Comment:
[P2] Preserve definite rejection semantics for REST commits
This ownership check also runs for RESTSnapshotCommit, whose `success=false`
is a server rejection; HttpClient::send_post sends once and does not replay
POSTs. REST snapshots can be catalog-only
(`SnapshotManager::get_latest_snapshot` uses RESTEnv), so the proposed snapshot
need not have a JSON object at this storage path. The resulting
SnapshotNotExist becomes publication_error, skips generated-DV cleanup below,
and marks a known rejection as an unknown outcome.
Reproduced through a real REST mock with a fixed-bucket PK lookup/DV table:
prepare/execute compaction at snapshot 3, expose a catalog-only snapshot 4 with
unchanged data, then reject publication of snapshot 5; commit.max-retries=0.
Merge-base 1899cfde returns a normal conflict and leaves no new DV files. This
head wraps the rejection as `Commit outcome may be unknown; retain prepared
files...` (source SnapshotNotExist{5}) and leaves a new `index/index-<uuid>-1`
container. A normal successful retry can hide the error but still cannot clean
that abandoned attempt's DV.
Please distinguish definite provider rejection from ambiguous storage
conditional-write replay, or verify ownership through the authoritative
provider. Keep the conservative retention for genuinely unknown outcomes and
cover catalog-only snapshots with a rejected DV compaction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]