JingsongLi opened a new pull request, #1090: URL: https://github.com/apache/paimon-rust/pull/1090
## Purpose Fix the native writer failure reported by apache/paimon#10482. Its Native CI run has 149 failures and 195 setup errors, all caused by rejecting otherwise valid Arrow FixedSizeList inputs for VECTOR fields. The shared input normalizer already canonicalizes ARRAY and MAP child aliases, but omits FixedSizeList. PyArrow's child field differs from Paimon's `element` field, so vectors fail schema validation before writing. ## Changes - Normalize fixed-size vector child fields to the table schema and reuse the payload buffers, including sliced inputs. - Retain dimension and element-type validation, following Java ParquetRowDataWriter's positional vector elements and vector-length checks. - Validate element nullability even when the input schema already matches. NULL vectors and NULL containing ROWs mask their hidden elements. - Add core regression tests for aliases, slices, buffer reuse, NULL parents, invalid dimensions/types, and foreign inputs carrying visible NULL elements under NOT NULL declarations. ## Validation - Reproduced the failure with the existing PyPaimon batch_vector_lookup_test.py and the new vector write regression test before the fix. - Core normalizer: 19 tests passed; TableWrite: 77 passed; DataEvolutionWriter: 18 passed. - Rebuilt the Python extension and passed 24 REST vector roundtrips covering Append/Data Evolution, Batch/Stream, FLOAT/DOUBLE, child aliases, sliced input, NULL vectors and NULL elements. - Rust 1.98.0 Clippy for paimon and pypaimon_rust, all targets with warnings denied, passed; formatting and diff checks passed. The Python counterpart continues to install Rust main. This fix must merge before its Native CI can pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
