iemejia opened a new pull request, #3859: URL: https://github.com/apache/avro/pull/3859
## What is the purpose of the change A `bytes` or `string` value is encoded as a length prefix followed by that many bytes of data, and an `array` or `map` block is encoded as an element count followed by that many items. A malicious or truncated input can declare a very large length or count while carrying little or no actual data, which causes a correspondingly large allocation before the shortfall is noticed. This applies the equivalent of the Java SDK fix [AVRO-4241](https://issues.apache.org/jira/browse/AVRO-4241) to the C++ SDK: when the source can report how many bytes remain, a declared length (or a collection block count) that exceeds the bytes actually available is rejected before allocating for it. The collection check uses the minimum on-wire size of the element schema, so a zero-byte element type (such as `null`) is never falsely rejected. Sources that cannot report their remaining size are unaffected. Adds `remainingBytes()` to `InputStream` (default -1) overridden by the memory input streams and exposed via `StreamReader`, and `bytesRemaining()` to the `Decoder` interface, overridden by `BinaryDecoder` and forwarded by the validating and resolving decoders. `decodeString`/`decodeBytes` and `GenericReader`'s array/map decoding consult it. The collection check is skipped under schema resolution, where the datum (reader) type may be larger than the wire (writer) type. This is a sub-task of [AVRO-4292](https://issues.apache.org/jira/browse/AVRO-4292) and resolves [AVRO-4294](https://issues.apache.org/jira/browse/AVRO-4294). ## Verifying this change This change added tests and can be verified as follows: - Added `lang/c++/test/AvailableBytesTests.cc` covering over-limit `string`/`bytes`/`array`/`map` rejection, within-limit values, and an array of nulls that must not be falsely rejected. - Run: `cd lang/c++ && mkdir build && cd build && cmake .. && make && ctest` ## Documentation - Does this pull request introduce a new feature? (no — hardening / robustness) - If yes, how is the feature documented? (not applicable) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
