JingsongLi opened a new pull request, #1231: URL: https://github.com/apache/paimon-rust/pull/1231
### Purpose Two concurrent writers can both observe that a snapshot is absent, overwrite the same snapshot path, and both report success. Replace check-then-rename/write publication with atomic creation that never replaces an existing snapshot. ### Brief change log - Add a FileIO atomic publication primitive and an optional FileIOProvider override. Use staged no-replace publication for the built-in local filesystem and FUSE, conditional rename where supported, and conditional creation for supported whole-object stores. - Treat an existing target as a conflict and preserve errors without falling back to an unconditional write. Backends without a reliable atomic primitive reject filesystem snapshot publication and require a catalog or provider implementation. - Keep snapshot serialization and LATEST hints in SnapshotManager, and retain the existing TableCommit conflict checks, retries, and uncertain-outcome recovery. - Add deterministic concurrent-writer, row-ID rebase, provider-error, unsupported-backend, and rename-error regressions; adapt filesystem fixtures to use safe publication. ### Tests - Paimon tests with `fulltext,vortex`: 4,704 unit tests and 702 integration tests passed; 7 existing tests ignored. All 73 integration targets were completed. - `cargo fmt --all -- --check`, explicit rustfmt validation for the included recovery tests, and `git diff --check` passed. - Strict workspace Clippy is blocked by four existing `clippy::nonminimal_bool` findings in unchanged `de_vector_read.rs`, `table_merge_into.rs`, and `vector_search_common.rs`. Workspace Clippy passed with `-D warnings -A clippy::nonminimal_bool`. - Real cloud endpoints and JindoSDK were not exercised. ### API and Format - Add a default optional `FileIOProvider::write_atomic_if_not_exists` method. Existing providers retain their path resolution; providers can explicitly supply atomic publication when their operator cannot. - Snapshot serialization and the on-disk format are unchanged. Custom filesystem operators without atomic complete-file publication now return an unsupported I/O error. Jindo remains read-only. ### Documentation Document atomic snapshot publication, provider overrides, FUSE requirements, and unsupported backend behavior in the getting-started guide and Rust API comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
