iemejia commented on code in PR #3850:
URL: https://github.com/apache/avro/pull/3850#discussion_r3564663018
##########
lang/py/avro/codecs.py:
##########
@@ -139,7 +200,11 @@ def compress(data: bytes) -> Tuple[bytes, int]:
def decompress(readers_decoder: avro.io.BinaryDecoder) ->
avro.io.BinaryDecoder:
length = readers_decoder.read_long()
data = readers_decoder.read(length)
- uncompressed = bz2.decompress(data)
+ limit = _max_decompress_length()
+ decompressor = bz2.BZ2Decompressor()
+ uncompressed = decompressor.decompress(data, limit + 1)
+ if len(uncompressed) > limit:
+ _raise_decompression_too_large(limit)
return avro.io.BinaryDecoder(io.BytesIO(uncompressed))
Review Comment:
Good catch. Reworked `BZip2Codec.decompress` to verify the decompressor
reached end-of-stream and to drain any trailing data, looping over concatenated
bzip2 streams so a multi-stream block is fully decoded. Added a test for a
truncated block. Pushed in 9f3c49d.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
