JingsongLi opened a new pull request, #451: URL: https://github.com/apache/paimon-rust/pull/451
## Summary Adds deletion-vector-backed DELETE support for data evolution tables, including read compatibility for DV-filtered raw and column-merged files. It also makes vector and full-text global-index search results respect live deletion vectors before final top-k/range calculation. ## Changes - Implemented `DataEvolutionDeleteWriter` to group `_ROW_ID` deletes by data-evolution anchor file, merge existing DVs, and write Java-compatible `DELETION_VECTORS` index files. - Applied deletion vectors in data-evolution reads for raw files and merged normal/vector/blob file groups, including `_ROW_ID` attachment and merged row counts. - Added DataFusion `DELETE` and `MERGE INTO ... WHEN MATCHED THEN DELETE` support for data-evolution tables gated by `deletion-vectors.enabled`. - Filtered vector and full-text global-index candidates using live DV metadata before final result computation. ## Testing - [x] `cargo check -p paimon` - [x] `cargo check -p paimon --features fulltext` - [x] `cargo check -p paimon-datafusion` - [x] `cargo check -p paimon-datafusion --features fulltext` - [x] `cargo test -p paimon-datafusion --test delete_tests -- --nocapture` - [x] `cargo test -p paimon-datafusion --test merge_into_tests -- --nocapture` - [x] `cargo test -p paimon data_evolution --lib -- --nocapture` - [x] `cargo test -p paimon deletion_vector --lib -- --nocapture` - [x] `cargo test -p paimon --features fulltext test_search_result_filters_deleted_row_ranges --lib -- --nocapture` - [x] `git diff --check` ## Notes Data-evolution DELETE requires `deletion-vectors.enabled = true`; tables without DV enabled continue to reject DELETE. Relates to apache/paimon#8322, apache/paimon#8380, apache/paimon#8426, and apache/paimon#8431. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
