iemejia commented on code in PR #3850:
URL: https://github.com/apache/avro/pull/3850#discussion_r3564493636
##########
lang/py/avro/codecs.py:
##########
@@ -123,7 +173,16 @@ def decompress(readers_decoder: avro.io.BinaryDecoder) ->
avro.io.BinaryDecoder:
data = readers_decoder.read_bytes()
# -15 is the log of the window size; negative indicates
# "raw" (no zlib headers) decompression. See zlib.h.
- uncompressed = zlib.decompress(data, -15)
+ limit = _max_decompress_length()
+ decompressor = zlib.decompressobj(-15)
+ # Request at most limit + 1 bytes so that an over-large block is
detected
+ # without allocating the whole (potentially huge) output.
+ uncompressed = decompressor.decompress(data, limit + 1)
+ if len(uncompressed) > limit:
+ _raise_decompression_too_large(limit)
+ uncompressed += decompressor.flush()
+ if len(uncompressed) > limit:
+ _raise_decompression_too_large(limit)
Review Comment:
Fixed in 638aac57cb — DeflateCodec.decompress now accumulates into a
bytearray, so flush() output is appended in place instead of creating an extra
full-size copy of the decompressed data.
##########
lang/py/avro/codecs.py:
##########
@@ -184,6 +256,8 @@ def decompress(readers_decoder: avro.io.BinaryDecoder) ->
avro.io.BinaryDecoder:
if not chunk:
break
uncompressed.extend(chunk)
+ if len(uncompressed) > limit:
+ _raise_decompression_too_large(limit)
Review Comment:
Fixed in 638aac57cb — the zstandard path now checks `len(uncompressed) +
len(chunk) > limit` before extending, so the buffer never grows past the limit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
