emecii opened a new pull request, #11044: URL: https://github.com/apache/arrow-rs/pull/11044
# Which issue does this PR close? Closes #7124. # Rationale for this change Validated IPC decoding uses `ArrayDataBuilder`, which constructs a `BooleanBuffer` before checking whether the validity buffer is large enough. Malformed input can therefore panic instead of returning an `ArrowError`. `ArrayData::try_new` already performs the required check, but its direct builder callers do not benefit from it. # What changes are included in this PR? Move the existing bounds check into the builder's validated path, before `BooleanBuffer` construction. Keep the error wording, checked length-plus-offset arithmetic, and unsafe skip-validation behavior. Add builder regressions for short/empty buffers, offsets, overflow, explicit/inferred null counts and valid inputs. Add public record-batch decoder and `StreamReader` regressions using directly encoded malformed IPC, without constructing invalid Arrow arrays with unsafe code. # Are these changes tested? The short-buffer and overflow builder regressions and both IPC regressions fail with panics before the fix, then pass afterward. Local CPU validation on Rust 1.98.0 / macOS aarch64: - `cargo test --locked --workspace`: 6,934 tests/doctests passed, 29 ignored. - Separate CI-style `cargo test --locked -p <crate> --all-features` runs for `arrow-data`, `arrow-array` and `arrow-ipc`: 1,200 passed, 3 ignored. - Release-mode `arrow-data` and `arrow-ipc` library/integration tests: 201 passed. - Both new IPC regressions also pass with `arrow-data/force_validate` and `arrow-array/force_validate` enabled. - Workspace-wide all-target/all-feature Clippy with `-D warnings`, formatting, Typos 1.37.0 and diff whitespace checks passed. An additional combined all-feature run across those three crates fails in six existing IPC tests that intentionally construct invalid arrays. The same six failures reproduce on unchanged main when `force_validate` is unified into that test run; the separate commands used by CI pass. The existing IPC `StreamReader/read_10` benchmark was exercised on main and this patch. Shared-host timings varied substantially between repetitions, so no performance conclusion is claimed. # Are there any user-facing changes? An undersized validity buffer now produces a recoverable error through the validated builder/IPC path instead of a panic. No public API signature changes. Implementation and regression tests generated with OpenAI Codex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
