JingsongLi opened a new pull request, #10666: URL: https://github.com/apache/paimon/pull/10666
### Purpose Complete the remaining verified Native write/update nullability gaps for existing PyPaimon APIs, using current Java semantics. Depends on https://github.com/apache/paimon-rust/pull/1179. Native CI continues building Rust main; keep this PR draft until the Core fixes land. ### Changes - Match Java's separate nullable KV value type for non-cross-partition deduplicate tables. Valid DELETE records can omit non-key/non-sequence NOT NULL values without changing the logical schema. - Apply logical row-kind-specific NULL checks before optional filtering on Arrow and InternalRow writes. Preserve strict INSERT/UPDATE and other-engine/cross-partition rules. - Use the physical value schema for Arrow/Pandas transport and Parquet system-field output, preserving keys, sequence fields, nested constraints and metadata. - Validate actual selected row-ID update output against its declared NOT NULL fields, after matching/deduplication/merging, and restore physical root field flags without casting hidden children. - Apply the same target constraint to dedicated BLOB output, which bypasses the ordinary Arrow writer. Nullable Blob values and NULL children of non-NULL containers remain valid. - Add batch/stream Native regressions for upsert/MERGE source NULLs that Core must discard, backed by the companion Rust transport fix. Python Native APIs remain thin wrappers. ### Verification - New retained-delete and adjacent writer regressions: 237 passed, 121 skipped locally. Actual commits/readback, sequence ordering, physical schema, required fields and existing writer APIs remain covered. - New update-source and target-nullability matrix: 21 passed, 21 Native skipped. Covers Arrow/GenericRow upserts, MERGE, unused source columns, discarded duplicates, selected invalid NULLs, nullable targets, dedicated Blob and Blob collection children. - Update/upsert/MERGE/chunked writer suite, including restoration of NOT NULL physical root fields and the existing RecordBatch columns protocol. - Final changed Python files pass repository Flake8, Python 3.6 syntax parsing and diff checks. - Java's existing deduplicate-DELETE/sequence reference tests: 2 passed. - Companion Rust retained-delete and filtered-write suites, 103 Python-binding Rust tests, core library, Rust 1.98.0 all-target Clippy and rustfmt. - Reproductions first fail on the old implementation, then pass. Local Python tests did not rebuild or install the Native extension; Native cases remain enabled for CI after the Rust fix lands. No new dependency, public API, extension-version fallback or implicit commit-message deletion is added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
