JingsongLi commented on code in PR #10161:
URL: https://github.com/apache/paimon/pull/10161#discussion_r4179991406
##########
paimon-python/pypaimon/read/reader/partial_update_merge_function.py:
##########
@@ -92,16 +100,19 @@ def reset(self) -> None:
def add(self, kv: KeyValue) -> None:
row_kind_byte = kv.value_row_kind_byte
if not RowKind.is_add_byte(row_kind_byte):
- # DELETE / UPDATE_BEFORE require ignore-delete or
- # partial-update.remove-record-on-delete to be enabled,
- # and neither option is implemented in pypaimon yet, so
- # refuse the row rather than silently swallow it.
+ if self._ignore_delete:
+ # ignore-delete: drop the retract row and keep merging the
+ # rest of the group, as Java PartialUpdateMergeFunction does.
+ return
Review Comment:
[P2] Preserve Java initialization when the first legacy record is a retract
Java PartialUpdateMergeFunction initializes the accumulator from the first
retract before applying ignore-delete (add lines 155–164 / initRow),
specifically for retracts persisted by 0.7-era writers. This branch returns
without that initialization. If a legacy DELETE has value=100 and a later
INSERT/UPDATE_AFTER leaves that nullable field unset, Java preserves 100 but
Python now returns null. I reproduced this through two valid overlapping
level-0 KV files (one record per key in each file), normal
CommitMessage/manifest/snapshot commit, and actual table reads: both engines
read the same committed table; Java returns {id:1,value:100}, this head returns
{id:1,value:null}. A retract-only key remains absent in both. A source-free
Python control adding only first-retract accumulator initialization restores
the Java result. Please mirror Java initialization while still keeping
retract-only groups absent, and cover retract-first plus later partial-add
across separate overlapping files.
The new fixture covers add-then-retract and retract-only, which cannot reveal
the lost field value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
