dragonknightk opened a new issue, #1430:
URL: https://github.com/apache/paimon-rust/issues/1430

   ### Search before asking
   
   - [X] I searched in the 
[issues](https://github.com/apache/paimon-rust/issues) and found nothing 
similar.
   
   ### Description
   
   With `partial-update.remove-record-on-delete=true`, the Rust partial-update 
merge only handles DELETE and skips a lone UPDATE_BEFORE (-U), so a key that 
ends with a lone -U still reads its old row. Java master does the same today. 
[apache/paimon PR 11178](https://github.com/apache/paimon/pull/11178) (open) 
changes Java so that this key reads no row, and Rust needs the same rule before 
that ships, or the two readers return different rows from the same files.
   
   The gap has been reproduced against that PR: a fixed-bucket Parquet 
partial-update table with `partial-update.remove-record-on-delete=true` 
contains a committed `+I(1,10,20)` followed by a lone `-U(1,10,NULL)`. Java 
classes compiled from the PR head return zero rows, while `pypaimon-rust 0.5.0` 
returns the stale `(1,10,20)` row with the Python fallback forbidden.
   
   The same merge code is in `v0.5.0` and in current `main`. `pypaimon-rust 
0.5.0` is the latest release on PyPI.
   
   Expected behavior, following the Java `PartialUpdateMergeFunction` in that 
PR:
   
   - A -U marks the row removed without re-initializing it and writes its own 
non-null fields, so a later +I or +U brings the row back with the fields of 
other writers.
   - A merge result removed only by -U is UPDATE_BEFORE, also after write 
pre-merge and compaction. If a DELETE re-initialized the row in the same merge, 
the result stays DELETE.
   - `partial-update.remove-record-on-sequence-group` and sequence groups keep 
their current behavior.
   
   I will open a PR with the change and tests. (๑•̀ㅂ•́)و✧
   
   ### Willingness to contribute
   
   - [X] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to