JingsongLi opened a new pull request, #1144: URL: https://github.com/apache/paimon-rust/pull/1144
### Purpose Keep PyPaimon's existing large-string upserts on Native execution, including strings used as match keys, partition fields and nested values. PyPaimon currently falls back because core write normalization rejects `LargeUtf8` for the canonical `Utf8` STRING representation. Testing this path also exposes a DE update failure for ROW/ARRAY/MAP types with required children: runtime read schemas relax nested nullability, while the partial-file writer uses the declared table schema. Interleaving against the runtime schema leaves incompatible child types in the output. ### Brief change log - Normalize `LargeUtf8` to `Utf8` in the existing core write normalizer. This is an Arrow layout conversion for Java/Paimon's STRING values, with checked offset bounds; it does not introduce implicit numeric or binary casts. - Reuse recursive normalization for nested strings and for the key encoding performed by core upserts. - Normalize merged DE update values back to their declared write layout before constructing partial-column batches. Preserve NULL parents while validating visible required children. - Add a core upsert round trip covering multiple batches, duplicate and NULL keys, updates to multiple target rows, absent columns on append, and stable row IDs. - Cover sliced strings, Unicode, empty values, nested layouts and visible versus hidden NULL children. ### Tests - `cargo +1.98.0 test --quiet --locked -p paimon --lib`: 4151 passed, 6 ignored. - `cargo +1.98.0 clippy --quiet --locked -p paimon --lib --tests -- -D warnings`. - `cargo +1.98.0 fmt --all --check`. - Built Python bindings and verified the corresponding PyPaimon execution paths: 30 new Native/Python parity cases passed; the related DE/update/BLOB regression selection passed 568 tests with 2 skips. - Independent review of input eligibility, nested constraints and failure paths. ### API and Format No new public interfaces, options, dependencies or storage formats. Java STRING values and the existing declared write constraints remain authoritative. ### Documentation No documentation changes are required. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
