Mr-Nobody-Anonymous opened a new pull request, #55687:
URL: https://github.com/apache/arrow/pull/55687
### Rationale for this change
In `cpp/src/arrow/ipc`, streaming decoding and decompression can be hardened
against malformed or untrusted buffers (GH-55669). This pull request
incorporates the hardening work originally proposed by @Awesome06 in #55671,
while resolving the build failures across the C++ matrix and addressing all
active review findings.
### What changes are included in this PR?
1. **Memory Safety and Bound Checks in MessageDecoder:**
- Replaces raw pointer arithmetic and manual memcpy with `std::span` and
checked offsets in `MessageDecoderImpl`.
- Validates input sizes against negative values in
`MessageDecoder::Consume`.
- Protects chunk reassembly against out-of-bounds copies.
2. **IPC Read Ceilings in IpcReadOptions:**
- Adds configurable `max_metadata_length`, `max_body_length`, and
`max_decompression_ratio` to `IpcReadOptions`.
- Rejects negative lengths in `CheckMetadataLength` and `CheckBodyLength`.
- Enforces configured limits in the shared `ReadRecordBatchInternal`
function so that direct public `ReadRecordBatch` entry points cannot bypass
bounds checks.
3. **FlatBuffers and Metadata Hardening:**
- Adds null checks for FlatBuffers tensor and custom metadata fields.
- Removes unconditional `metadata->reserve` in `GetKeyValueMetadata` to
prevent untrusted metadata counts from triggering premature large vector
allocations.
4. **CI Build and Test Fixes:**
- Fixes compiler deduction error on `read_write_test.cc:3579` by
separating `int64_t*` and `int32_t*` options in initializer lists.
- Uses public IPC APIs (`MessageReader::Open`, `ReadSchema`) in direct
record batch regression tests instead of internal `RecordBatchFileReader`
helpers.
### Are these changes tested?
Yes, comprehensive unit tests have been added to
`cpp/src/arrow/ipc/read_write_test.cc` covering:
- Malformed header tokens and negative continuation tokens.
- Negative read limit options validation.
- Metadata and body length limit enforcement in streaming and direct
record-batch reads.
- Decompression ratio limits.
### Are there any user-facing changes?
New optional fields in `IpcReadOptions`: `max_metadata_length`,
`max_body_length`, and `max_decompression_ratio`. Defaults remain `-1`
(unlimited), preserving backwards compatibility.
### Was AI used for this PR?
**PR code and description written by:**
- [x] AI
**Reviewed before submission by:**
- [x] Human
* GitHub Issue: GH-55669
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]