JingsongLi opened a new pull request, #1045: URL: https://github.com/apache/paimon-rust/pull/1045
### Purpose Prevent abort and failure cleanup from deleting prepared files which a successful commit may already reference. Storage/catalog publication can succeed while its response fails, and even a later validation error does not prove that a message's files are uncommitted. This follows the conservative retention policy in apache/paimon #10399: retain orphan files rather than risk deleting published data. ### Brief change log - Remove message-based data, changelog, sidecar and index deletion from `TableCommit::abort`. Preserve the public method as a no-op, including for Format Tables and the C/Go/Python wrappers. - Remove automatic abort calls from guarded commits, partial write/delete preparation, update/upsert failure handlers, and sorted/Lumina/vector index builds. Add comments prohibiting file deletion based on CommitMessage. - Reset row-id update state without deleting its prepared files; remove the commit-user state and helper that only existed for cleanup. - Preserve Format Table staging after validation failures and retain already published targets after partial publication or partition-registration response errors. Successful publication can still discard its staging copies. - Keep cleanup of outputs still owned by a writer and pre-publication temporary manifests. Their ownership is established without submitted messages. - Update tests and Rust/Python/Go documentation to describe retention after abort. ### Tests Regression cases cover: - Errors before or after actual snapshot publication, followed by repeated explicit abort and identity-based retry. - An already published global index followed by a stale guarded retry. - Data/changelog/index/BLOB paths, external and relocated paths, sidecars, deletion vectors and compaction increments. - Published and unpublished row-id updates, late grouped input/callback failures, and partial partition preparation. - Later sorted/vector index shard failures preserving the completed first shard without registering an incomplete index. - Format Table validation retry, partial publication, and a real HTTP catalog which registers the partition before returning an undecodable response; published rows remain readable. - Sixteen Python binding combinations: batch/stream, append/primary-key, local/serialized messages and prepared/published files. The baseline failed all sixteen cases because abort removed the files. Verification completed locally: - `cargo test --locked -p paimon --lib` plus the selected update, external/relocated path, deletion-vector, BLOB collection and writer resource integration suites: 3,721 core tests and 60 integration tests passed (six pre-existing core tests ignored). - The final added lost partition-registration response regression passed separately. - Rebuilt Python binding: all 96 tests in `test_table_commit.py`, `test_write.py` and `test_update.py` passed, including all sixteen file-preservation cases. - Independent source review found no remaining submitted-message deletion path. - `cargo fmt --all -- --check`, Python syntax checks and `git diff --check` passed. - `cargo clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings` passed. ### API and Format Public commit and abort signatures and storage formats are unchanged. `abort(messages)` now preserves files instead of deleting them; abandoned prepared writes may leave orphan files. A failed Format Table append can retain outputs already visible to readers. ### Documentation Update getting-started guidance, Python README/type stubs and Go Abort comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
