JingsongLi opened a new pull request, #10242: URL: https://github.com/apache/paimon/pull/10242
### Purpose Enable Native ARRAY/MAP/ROW updates and upserts for ordinary Parquet data-evolution tables. The previous guards fall back to Python whenever updated columns are nested, including when the core can execute the complete operation. This change removes those guards after completing the required nested normalization and coercion in Rust core. **Depends on https://github.com/apache/paimon-rust/pull/973.** Keep this PR as a draft until the Rust change is merged. Native CI continues to build `apache/paimon-rust@main`; it is not redirected to a fork or feature branch. - Enable direct/grouped row-ID updates, incremental row-ID writers, predicate assignments and key upserts for nested columns. - Keep conversion, matching and update execution in Rust core, with the existing Python wrappers. - Add end-to-end cases for batch/stream updates, literal/Arrow/callable assignments, Arrow/row upserts, duplicate source keys, appends, untouched fields, slices, chunked arrays, empty collections, parent/child NULLs and failed grouped-update cleanup. - Assert that enabled cases cannot silently fall back to Python, and verify committed results through both Python and Native readers. ### Tests Built the extension from the dependent Rust branch and ran these suites with PyArrow 18.1 and all Native CI flags enabled: ``` pypaimon/tests/native_nested_update_test.py pypaimon/tests/native_update_test.py pypaimon/tests/table_update_test.py pypaimon/tests/table_update_by_row_id_chunked_test.py pypaimon/tests/native_write_capabilities_test.py pypaimon/tests/native_write_test.py ``` Result: **779 passed, 2 skipped, 16 subtests passed**. The two skips are Python-only MAP view-key cases where PyArrow 18.1 lacks an Arrow `take` kernel; their Native counterparts pass. Native coverage counters: row-ID 167, grouped 54, predicate 77, upsert 21, incremental 15. Changed Python files pass flake8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
