Awesome06 opened a new pull request, #55671: URL: https://github.com/apache/arrow/pull/55671
### Rationale for this change In `arrow::ipc::MessageDecoderImpl`, chunk reassembly relied on unchecked pointer arithmetic (`memcpy(out + offset, ...)`) during streaming message reconstruction. Furthermore, `MessageDecoder::Consume(data, size)` did not validate that `size >= 0`, which could construct invalid negative-sized `Buffer` instances and corrupt internal chunk accounting. In addition, FlatBuffers tensor metadata decoders called `.size()` directly on `tensor->shape()` without verifying that the optional vector was non-null, and IPC readers lacked opt-in limits to constrain untrusted metadata sizes, body lengths, or decompression expansion ratios. ### What changes are included in this PR? - Replaces raw pointer arithmetic in `MessageDecoderImpl::ConsumeDataChunks` with `std::span` and checked addition via `::arrow::internal::AddWithOverflow`. - Adds `DCHECK_GE` invariants on chunk sizing in `ConsumeDataChunks`. - Enforces validation in `MessageDecoder::Consume` rejecting `size < 0` at the public API entry point. - Adds `CHECK_FLATBUFFERS_NOT_NULL` guards for `tensor->shape()`, dimension elements, and custom metadata pairs in `metadata_internal.cc`. - Adds `max_metadata_length`, `max_body_length`, and `max_decompression_ratio` to `IpcReadOptions` (defaulting to `-1` for unlimited / full backwards compatibility). - Enforces `max_decompression_ratio` in `DecompressBuffer` using `MultiplyWithOverflow` and slices compressed payloads with `std::span`. - Plumbs `IpcReadOptions` through `MessageDecoder` constructors and `StreamDecoderImpl`. ### Are these changes tested? Yes. Existing IPC streaming and file tests pass. New deterministic unit tests were added in `cpp/src/arrow/ipc/read_write_test.cc`: - `TestMessageDecoderBounds.NegativeConsumeSize` - `TestMessageDecoderBounds.NegativeContinuationToken` - `TestMessageDecoderBounds.NegativeMetadataLength` - `TestMessageDecoderBounds.MaxMetadataLengthExceeded` - `TestStreamDecoderLimits.MaxBodyLengthExceeded` - `TestDecompressBufferLimits.NegativeUncompressedSize` - `TestDecompressBufferLimits.ZeroCompressedPayloadWithNonZeroUncompressed` - `TestDecompressBufferLimits.MaxDecompressionRatioExceeded` - `TestMetadataBounds.EmptyOrInvalidTensorMetadata` ### Are there any user-facing changes? No breaking changes. Users can optionally configure `max_metadata_length`, `max_body_length`, and `max_decompression_ratio` on `IpcReadOptions` when reading IPC streams. ### Was AI used for this PR? PR code and description written by: - [x] Human - [x] AI Reviewed before submission by: - [x] Human - [x] AI - [ ] Not reviewed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
