JingsongLi opened a new pull request, #10411: URL: https://github.com/apache/paimon/pull/10411
## Purpose Use Native for capabilities already supported by PyPaimon: column-subset append writes, column-subset upserts, and ordinary FLOAT/DOUBLE upsert keys. Depends on https://github.com/apache/paimon-rust/pull/1054 merging first. The Native CI job continues to build Rust from `main`. ## Changes - Forward append `with_write_type` to the core batch/stream writer. Accept both selected-column and full-table Arrow inputs, projecting full inputs before writing. Pandas and row inputs use the selected schema. - Preserve selected columns when a later advanced API selects the Python writer; keep PK setters on the Python path before any data is staged. - Send homogeneous partial row inputs to core upsert without replacing absent fields with NULL. Enable ordinary FLOAT/DOUBLE key matching. - Fix Python partition/bucket extraction to resolve columns in the actual input schema. Its full-schema indices previously routed partial Arrow input to the wrong partition or bucket. - Add end-to-end tests for batch/stream writes, Arrow/Pandas/row inputs, both grouping paths, fractional and NULL keys, duplicate-key semantics, missing columns, omitted BLOB values and dedicated-only partial files. Physical write columns, dedicated-writer selection, and existing row-ID ranges follow Java append/data-evolution semantics. ## Verification - Native write/update/upsert capabilities and data-evolution suite against local Rust bindings: 609 passed. - Full Native CI test scope: 8684 passed, 275 skipped, 1439 subtests passed. One full-input/partial-writer failure found there is fixed and passes in the focused rerun. - The remaining 12 failures in that broad run compare positive and negative NaN sequence ties. Their comparator is unchanged by this PR; NaN behavior is explicitly outside the requested scope. - Rust core: 3737 passed, 6 ignored; partial-write/update integration tests: 37 passed. - Rust Python binding suite: 313 passed. - Changed Python files pass the repository flake8 configuration. Kept as a draft until the Rust prerequisite merges into `main`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
