JingsongLi opened a new pull request, #915: URL: https://github.com/apache/paimon-rust/pull/915
### Purpose Close the remaining TableCommit correctness gaps found after #913, including the batch recovery API added by #912. Replaying an expired commit identity can currently add an already active file again and inflate snapshot row counts. Deletion-vector commits also reject unrelated writes and cannot prepare a DELETE when a REST catalog supplies snapshots without physical snapshot files. ### Brief change log - Validate pending data, changelog, sidecar, and index files before recovery, and force append conflict checks for both recovery APIs. Normalize empty partition identifiers and reject duplicate identifiers in a recovery batch. - Validate deletion-vector replacements and referenced data against each commit attempt. Allow unrelated concurrent writes while rejecting stale replacements and dangling references, including after publication retries. - Pin the catalog snapshot when preparing DELETE and reuse supplied snapshots during row-ID validation. - Treat empty unpartitioned overwrite as a table overwrite, and preserve existing global indexes for ordinary dedicated-storage INSERT files. ### Tests - Regression coverage for expired identities, all recovered file locations, batch validation before publication, empty overwrite, dedicated INSERTs, concurrent deletion vectors, publication retries, and actual REST DELETE/read round trips without physical snapshot files. - Existing no-manifest-merging regression remains covered. - `cargo test --locked -p paimon --all-targets --features fulltext,vortex`: 3429 passed, 6 ignored. - `pytest -q bindings/python/tests/test_table_commit.py`, using a wheel built from this branch: 36 passed. - `cargo clippy --locked -p paimon --all-targets --features fulltext,vortex -- -D warnings`: passed. - `cargo fmt --all -- --check` and explicit rustfmt check of the included recovery test file: passed. The workspace-wide Clippy command is blocked by an existing `clippy::collapsible_match` warning at `crates/integrations/datafusion/src/partition_count_pushdown.rs:165`, which is unchanged from main. ### API and Format No public signatures or storage formats change. Recovery rejects missing or already active files and duplicate checkpoint identifiers; valid recovery and unrelated concurrent DELETE operations succeed. ### Documentation No new configuration or documentation changes are required. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
