JingsongLi commented on PR #10161: URL: https://github.com/apache/paimon/pull/10161#issuecomment-5952248525
Reviewed head 706417553f. The partial-update implementation has end-to-end value, and I did not find an introduced read-semantic defect. I verified actual Java-produced Parquet and Avro fixtures containing both DELETE and UPDATE_BEFORE: each manifest records two retract rows, Python uses the merge path, retains the earlier insert, drops a retract-only key, and merges later non-null fields exactly like Java. Full reads, projections, and canonical Python boolean spellings passed. Disabling only the new retract-skipping branch makes these fixtures fail. **[P2] The new persisted-retraction test still removes the retracts on the write side** (`native_write_capabilities_test.py:264–276`). It writes with `ignore-delete=true`. The exact Native CI dependency (`paimon-rust` 6267e9c0) generates row kinds and then applies `RowKindFilter` before persisting; DELETE/UPDATE_BEFORE are discarded. Consequently the Python reader only sees the insert, and the assertion does not exercise the new `PartialUpdateMergeFunction.add` branch. Reverting the option-support guard and getting `NotImplementedError` proves table admission, but does not prove retract handling. The same physical filtering also explains why the aggregation variant passes. Please persist a legacy/equivalent fixture whose physical `_VALUE_KIND` contains 1 and 3, assert positive manifest `deleteRowCount`, and read it with Python. Include the existing-key and retract-only-key cases, and verify that removing only the new skip branch fails the test. Modern Java/native writers also filter these rows, so simply switching writer languages while retaining `ignore-delete=true` will not create that fixture. Validation: 71 relevant Python tests and 47 Java partial-update tests passed; 25 native-extension cases were skipped locally, while the exact-head Native CI is green. The separate persisted-file cross-engine checks above passed; configured Python lint, Python 3.6 grammar, and diff checks passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
