JingsongLi commented on PR #8263: URL: https://github.com/apache/paimon/pull/8263#issuecomment-4747350087
I think this still breaks commit conflict handling on object stores. The latest snapshot is read before acquiring the commit lock, and `newSnapshotPath` is computed from that value. Previously, the `!fileIO.exists(newSnapshotPath)` check inside `lock.runWithLock` was the stale-snapshot guard: if another writer committed the same snapshot id while this writer was waiting for the lock, this returned false and `FileStoreCommitImpl` retried with the newer latest snapshot. With the new object-store branch, we skip that check and then write `snapshot-N` with `overwrite=true`, so two serialized object-store commits that both planned snapshot N can both return success, with the second one replacing the first snapshot file. That can lose a committed snapshot even when the external lock is working. We need to preserve the no-overwrite/conflict signal for an existing snapshot file without requiring ListBucket, rather than blindly overwriting snapshot metadata on object stores. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
