fallenmi opened a new pull request, #50884: URL: https://github.com/apache/arrow/pull/50884
### Rationale for this change The IPC writer selected the top-level validity-buffer layout from the logical extension type. Extension arrays backed by null or union storage therefore received an extra validity-buffer entry, while the reader reconstructed the physical storage layout without that entry. The resulting buffer shift produced invalid data and could lead to a crash when values were accessed. Fixes #50623. Fixes #15068. ### What changes are included in this PR? - Select the IPC validity-buffer layout from `DataType::storage_id()`, which is unchanged for ordinary Arrow types and resolves to the physical type for extensions. - Add roundtrip regressions for extension arrays backed by dense union, sparse union, and null storage. - Exercise both IPC metadata versions V4 and V5. The null-storage test includes an adjacent `int64` column so a shifted buffer is detected directly. ### Are these changes tested? - On unpatched current `main`, both new regressions failed with `Buffer #1 too small` validation errors. - Focused patched regressions: 2/2 passed. - Full `arrow-ipc-read-write-test`: 412/412 passed with the official `arrow-testing` data. - `clang-format` 20.1.8 dry run and `git diff --check`: passed. ### Are there any user-facing changes? Yes. IPC file and stream roundtrips for extension types with null or union storage now preserve valid buffer layouts. There is no public API or ABI change. **This PR contains a Critical Fix.** It prevents the IPC writer from producing invalid data for valid extension arrays and addresses the crash path described in #50623. ### AI-assisted contribution disclosure OpenAI Codex generated the patch and regression tests under the account owner authorization. Codex reproduced the issue on current `main`, reviewed the complete diff, audited visible ownership and overlap, and ran the validation listed above. This PR remains a Draft pending the account owner personal review before it is marked ready. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
