iemejia commented on code in PR #3850:
URL: https://github.com/apache/avro/pull/3850#discussion_r3567341958
##########
lang/py/avro/codecs.py:
##########
@@ -123,7 +173,18 @@ def decompress(readers_decoder: avro.io.BinaryDecoder) ->
avro.io.BinaryDecoder:
data = readers_decoder.read_bytes()
# -15 is the log of the window size; negative indicates
# "raw" (no zlib headers) decompression. See zlib.h.
- uncompressed = zlib.decompress(data, -15)
+ limit = _max_decompress_length()
+ decompressor = zlib.decompressobj(-15)
+ # Request at most limit + 1 bytes so that an over-large block is
detected
+ # without allocating the whole (potentially huge) output. Accumulate
into
+ # a bytearray so the flush() output is appended in place rather than
+ # creating an extra full-size copy of the already-decompressed data.
+ uncompressed = bytearray(decompressor.decompress(data, limit + 1))
+ if len(uncompressed) > limit:
+ _raise_decompression_too_large(limit)
+ uncompressed += decompressor.flush()
+ if len(uncompressed) > limit:
+ _raise_decompression_too_large(limit)
Review Comment:
Good catch. Added a `decompressor.eof` check after flush() — a
truncated/incomplete deflate block now raises `InvalidAvroBinaryEncoding`,
restoring the reject-truncated behavior that `zlib.decompress` had.
##########
lang/py/avro/codecs.py:
##########
@@ -158,7 +242,15 @@ def decompress(readers_decoder: avro.io.BinaryDecoder) ->
avro.io.BinaryDecoder:
# Compressed data includes a 4-byte CRC32 checksum
length = readers_decoder.read_long()
data = readers_decoder.read(length - 4)
+ limit = _max_decompress_length()
Review Comment:
Fixed — `SnappyCodec.decompress` now validates `length >= 4` and raises a
codec-specific `InvalidAvroBinaryEncoding` before reading `length - 4` bytes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
