timmyb32r opened a new pull request, #12256:
URL: https://github.com/apache/arrow-rs/pull/12256

   # Which issue does this PR close?
   
   None — standalone bug fix.
   
   # Rationale for this change
   
   `Schema::normalize` discards the nullability of expanded parent structs. A 
non-nullable child of a nullable struct therefore becomes a non-nullable 
flattened field, even though a null parent makes that child's normalized value 
null.
   
   For example:
   
   ```text
   Before:
   parent: nullable Struct<child: non-nullable Int32>
   
   Previously normalized:
   parent.child: non-nullable Int32
   
   Expected:
   parent.child: nullable Int32
   ```
   
   This can produce a schema that rejects the corresponding data from 
`RecordBatch::normalize`.
   
   # What changes are included in this PR?
   
   Propagate accumulated parent nullability during schema normalization and 
combine it with each field's own nullability.
   
   Fields retained at the depth limit also inherit nullability from expanded 
ancestors. Nested fields inside retained structs and list element nullability 
are preserved.
   
   # Are these changes tested?
   
   Added a regression test covering all combinations of parent, intermediate 
struct, and child nullability, with scalar and list fields and multiple depth 
limits. It also checks that nullability does not leak into unrelated sibling 
fields.
   
   Extended the nullable-struct record batch test to verify that 
`Schema::normalize` produces the expected schema and that this schema accepts 
the normalized columns.
   
   Both regressions fail before the fix and pass afterward.
   
   Validation passed:
   
   - `cargo test -p arrow-schema -p arrow-array --all-features`.
   - `cargo test --workspace`.
   - Targeted schema and record batch regression tests under Miri.
   - `cargo clippy --workspace --all-targets --all-features -- -D warnings`.
   - Formatting and spelling checks.
   
   # Are there any significant user-facing changes?
   
   Flattened fields now correctly become nullable when an expanded ancestor is 
nullable, including when normalization stops at a depth limit.
   
   There are no public API signature changes.
   
   # AI assistance
   
   Codex helped implement the fix and regression tests, run validation, and 
draft this PR description.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to