iemejia opened a new pull request, #3921:
URL: https://github.com/apache/avro/pull/3921
## What changes were proposed in this pull request?
The Trevni readers sized several allocations directly from values read from
the file header/metadata without validating them against the input actually
available. For a malformed, corrupted, or truncated file these counts/lengths
could greatly exceed the bytes present, driving oversized allocations, or
overflow to a negative size.
This adds a shared `InputBuffer.remaining()` /
`InputBuffer.checkLength(count, minBytesPerElement)` helper that rejects a
negative value and any value that could not be backed by the bytes remaining,
and applies it to the reader paths that size allocations from header/metadata
values:
- `ColumnFileReader.readHeader` — column count
- `ColumnDescriptor.ensureBlocksRead` — block count
- `InputBuffer.readBytes` / `readString` — length-prefixed byte arrays
- `ColumnValues.startBlock` — compressed block size; this path now also uses
`Math.addExact` for the `+ checksum.size()` addition to guard against integer
overflow.
Reading a malformed file now fails fast with a clear `IOException`; valid
files read unchanged.
## How was this patch tested?
- New test in `TestColumnFile`:
- `oversizedColumnCountIsRejected` — a file whose header column count is
overwritten with `Integer.MAX_VALUE` now fails fast instead of attempting a
large allocation.
- Full `trevni-core` module test suite passes.
### JIRA
- [AVRO-4325](https://issues.apache.org/jira/browse/AVRO-4325)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
