YiYao223 opened a new pull request, #11861: URL: https://github.com/apache/arrow-rs/pull/11861
# Which issue does this PR close? - Closes #11832 # Rationale for this change `RecordBatch::try_new` previously validated non-nullable fields using `c.null_count() > 0`, which only checked the top-level validity buffer. For arrays whose nulls reside in child arrays or dictionary values (such as `UnionArray`, `RunArray`, or `DictionaryArray`), `c.null_count()` is `0` even when logical nulls exist, causing `RecordBatch::try_new` to accept logical nulls in non-nullable columns. Other array constructors like `StructArray::try_new`, `ListArray::try_new`, and `FixedSizeListArray::try_new` already check logical nulls and reject these cases. # What changes are included in this PR? - Check `c.logical_null_count() > 0` in `RecordBatch::try_new` for non-nullable fields. - Update `RecordBatch::try_from_iter` to determine column nullability via `array.logical_null_count() > 0`. - Add unit tests verifying that `UnionArray`, `RunArray`, and `DictionaryArray` containing nulls are rejected in non-nullable `RecordBatch` columns, and that `try_from_iter` marks such columns nullable. # Are these changes tested? Yes, new unit tests were added in `arrow-array/src/record_batch.rs` covering `UnionArray`, `RunArray`, and `DictionaryArray` logical null verification in `RecordBatch::try_new` and `RecordBatch::try_from_iter`. # Are there any significant user-facing changes? Non-nullable fields in `RecordBatch::try_new` will now correctly return an error if passed columns containing logical nulls (aligning with `StructArray`, `ListArray`, and `FixedSizeListArray`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
