Awesome06 opened a new pull request, #55671:
URL: https://github.com/apache/arrow/pull/55671

   ### Rationale for this change
   
   In `arrow::ipc::MessageDecoderImpl`, chunk reassembly relied on unchecked 
pointer arithmetic (`memcpy(out + offset, ...)`) during streaming message 
reconstruction. Furthermore, `MessageDecoder::Consume(data, size)` did not 
validate that `size >= 0`, which could construct invalid negative-sized 
`Buffer` instances and corrupt internal chunk accounting. In addition, 
FlatBuffers tensor metadata decoders called `.size()` directly on 
`tensor->shape()` without verifying that the optional vector was non-null, and 
IPC readers lacked opt-in limits to constrain untrusted metadata sizes, body 
lengths, or decompression expansion ratios.
   
   ### What changes are included in this PR?
   
   - Replaces raw pointer arithmetic in `MessageDecoderImpl::ConsumeDataChunks` 
with `std::span` and checked addition via `::arrow::internal::AddWithOverflow`.
   - Adds `DCHECK_GE` invariants on chunk sizing in `ConsumeDataChunks`.
   - Enforces validation in `MessageDecoder::Consume` rejecting `size < 0` at 
the public API entry point.
   - Adds `CHECK_FLATBUFFERS_NOT_NULL` guards for `tensor->shape()`, dimension 
elements, and custom metadata pairs in `metadata_internal.cc`.
   - Adds `max_metadata_length`, `max_body_length`, and 
`max_decompression_ratio` to `IpcReadOptions` (defaulting to `-1` for unlimited 
/ full backwards compatibility).
   - Enforces `max_decompression_ratio` in `DecompressBuffer` using 
`MultiplyWithOverflow` and slices compressed payloads with `std::span`.
   - Plumbs `IpcReadOptions` through `MessageDecoder` constructors and 
`StreamDecoderImpl`.
   
   ### Are these changes tested?
   
   Yes. Existing IPC streaming and file tests pass. New deterministic unit 
tests were added in `cpp/src/arrow/ipc/read_write_test.cc`:
   - `TestMessageDecoderBounds.NegativeConsumeSize`
   - `TestMessageDecoderBounds.NegativeContinuationToken`
   - `TestMessageDecoderBounds.NegativeMetadataLength`
   - `TestMessageDecoderBounds.MaxMetadataLengthExceeded`
   - `TestStreamDecoderLimits.MaxBodyLengthExceeded`
   - `TestDecompressBufferLimits.NegativeUncompressedSize`
   - `TestDecompressBufferLimits.ZeroCompressedPayloadWithNonZeroUncompressed`
   - `TestDecompressBufferLimits.MaxDecompressionRatioExceeded`
   - `TestMetadataBounds.EmptyOrInvalidTensorMetadata`
   
   ### Are there any user-facing changes?
   
   No breaking changes. Users can optionally configure `max_metadata_length`, 
`max_body_length`, and `max_decompression_ratio` on `IpcReadOptions` when 
reading IPC streams.
   
   ### Was AI used for this PR?
   
   PR code and description written by:
   - [x] Human
   - [x] AI
   
   Reviewed before submission by:
   - [x] Human
   - [x] AI
   - [ ] Not reviewed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to