shyjsarah opened a new pull request, #573:
URL: https://github.com/apache/paimon-rust/pull/573
### Purpose
Linked issue: close #572
Paimon Rust currently rejects `ignore-delete` options for
`merge-engine=partial-update` tables and unconditionally fails on
`DELETE` / `UPDATE_BEFORE` rows. This prevents compatibility with Java Paimon
tables and historical files that use the supported ignore-delete semantics.
This change aligns the basic Rust partial-update implementation with Java for
this option while keeping advanced partial-update modes unsupported.
### Brief change log
- Accept exact `ignore-delete` and `partial-update.ignore-delete` options for
partial-update tables, including explicit `false`.
- Ignore `DELETE` and `UPDATE_BEFORE` during read-side partial-update merging
when enabled, including retract-only keys.
- Filter direct `_VALUE_KIND` retract rows before writer buffering, sequence
allocation, data-file merge, and input changelog generation.
- Reject changing the effective option from `true` back to `false`, including
alias removal and canonical-key override cases.
- Keep sequence groups, partial aggregation, remove-record options, and
deletion-vector combinations fail-fast.
### Tests
- [x] `cargo fmt --all -- --check`
- [x] `cargo test -p paimon --lib`
- [x] `cargo test -p paimon --test incremental_batch_scan_test`
- [x] `cargo test -p paimon-datafusion --test pk_tables`
- [x] `cargo clippy -p paimon --lib --tests -- -D warnings`
- [x] `cargo clippy -p paimon-datafusion --test pk_tables -- -D warnings`
Coverage includes:
- canonical and alias option parsing;
- explicit `false`;
- `DELETE` and `UPDATE_BEFORE`;
- retract-only keys;
- writer buffer, sequence, data, and input changelog behavior;
- ALTER attempts from effective `true` to `false`;
- core table and DataFusion SQL end-to-end reads.
### API and Format
No public API or storage-format change. The change only enables an existing
Java-compatible table option and filters retract rows according to that
option.
### Documentation
Updated the SQL table-option documentation with the supported keys, behavior,
ALTER restriction, and remaining partial-update limitations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
