iemejia opened a new pull request, #3930:
URL: https://github.com/apache/avro/pull/3930
## What is the purpose of the change
Java SDK implementation of AVRO-4303 (parent). The available-bytes guard
added
under AVRO-4292 rejects a declared bytes/string length that exceeds the data
remaining only when the decoder can report the number of bytes remaining
(memory-backed or seekable sources). On a non-seekable stream (socket, pipe,
decompression stream) the check is a no-op, so a huge declared length still
drives a single large up-front allocation before any payload is read. A tiny
truncated input can therefore force a multi-hundred-MB allocation.
When the remaining byte count is unknown, this reads the bytes/string value
into
a buffer that grows in bounded chunks rather than allocating the full
attacker-declared length up front. A truncated or hostile stream then fails
with
a bounded `EOFException` after a bounded allocation instead of an
`OutOfMemoryError`. The existing single-allocation fast path is kept when the
remaining byte count is known. The same fix is applied to
`DirectBinaryDecoder`,
which reads straight from a stream.
## Verifying this change
This change added tests and can be verified as follows:
- Added `TestBinaryDecoderBoundedRead`: a near-2GB declared bytes/string
length
on a truncated non-seekable stream fails with a bounded `EOFException`
(not an
`OutOfMemoryError`) for both the buffered `BinaryDecoder` and
`DirectBinaryDecoder`; a legitimately large value on a non-seekable stream
still round-trips; small values and seekable sources keep the direct path.
- Existing `TestBinaryDecoder` (68 tests) continues to pass.
## Documentation
- Does this pull request introduce a new feature? (no — DoS hardening)
- If yes, how is the feature documented? (JavaDocs on the new bounded-read
helper in `BinaryDecoder`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
