dragonknightk opened a new issue, #1430: URL: https://github.com/apache/paimon-rust/issues/1430
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon-rust/issues) and found nothing similar. ### Description With `partial-update.remove-record-on-delete=true`, the Rust partial-update merge only handles DELETE and skips a lone UPDATE_BEFORE (-U), so a key that ends with a lone -U still reads its old row. Java master does the same today. [apache/paimon PR 11178](https://github.com/apache/paimon/pull/11178) (open) changes Java so that this key reads no row, and Rust needs the same rule before that ships, or the two readers return different rows from the same files. The gap has been reproduced against that PR: a fixed-bucket Parquet partial-update table with `partial-update.remove-record-on-delete=true` contains a committed `+I(1,10,20)` followed by a lone `-U(1,10,NULL)`. Java classes compiled from the PR head return zero rows, while `pypaimon-rust 0.5.0` returns the stale `(1,10,20)` row with the Python fallback forbidden. The same merge code is in `v0.5.0` and in current `main`. `pypaimon-rust 0.5.0` is the latest release on PyPI. Expected behavior, following the Java `PartialUpdateMergeFunction` in that PR: - A -U marks the row removed without re-initializing it and writes its own non-null fields, so a later +I or +U brings the row back with the fields of other writers. - A merge result removed only by -U is UPDATE_BEFORE, also after write pre-merge and compaction. If a DELETE re-initialized the row in the same merge, the result stays DELETE. - `partial-update.remove-record-on-sequence-group` and sequence groups keep their current behavior. I will open a PR with the change and tests. (๑•̀ㅂ•́)و✧ ### Willingness to contribute - [X] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
