QuakeWang commented on code in PR #522:
URL: https://github.com/apache/paimon-rust/pull/522#discussion_r3592253578
##########
crates/paimon/src/table/table_commit.rs:
##########
@@ -1065,6 +1113,35 @@ impl TableCommit {
}
/// Check if this commit was already completed (idempotency).
+ async fn is_committed_identifier(
+ &self,
+ latest_snapshot: &Option<Snapshot>,
+ commit_identifier: i64,
+ ) -> Result<bool> {
+ let Some(latest) = latest_snapshot else {
+ return Ok(false);
+ };
+ let earliest_snapshot_id = self
+ .snapshot_manager
+ .earliest_snapshot_id()
+ .await?
+ .unwrap_or(latest.id());
+ for snapshot_id in (earliest_snapshot_id..=latest.id()).rev() {
+ let snapshot = if snapshot_id == latest.id() {
+ Some(latest.clone())
+ } else {
+ self.snapshot_manager.get_snapshot(snapshot_id).await.ok()
Review Comment:
This suppresses all snapshot read errors, including I/O and deserialization
failures. If the latest snapshot for this commit user cannot be read, filtering
may fall back to an older identifier and commit the same data again.
##########
crates/paimon/src/table/table_commit.rs:
##########
@@ -262,6 +291,7 @@ impl TableCommit {
},
None,
commit_identifier,
+ false,
Review Comment:
Only APPEND has a filtered retry path. OVERWRITE and TRUNCATE always disable
filtering, so retrying an uncertain result after an intervening commit can
re-execute the destructive operation and remove data written in between.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
