Copilot commented on code in PR #3850:
URL: https://github.com/apache/avro/pull/3850#discussion_r3564616023
##########
lang/py/avro/codecs.py:
##########
@@ -139,7 +200,11 @@ def compress(data: bytes) -> Tuple[bytes, int]:
def decompress(readers_decoder: avro.io.BinaryDecoder) ->
avro.io.BinaryDecoder:
length = readers_decoder.read_long()
data = readers_decoder.read(length)
- uncompressed = bz2.decompress(data)
+ limit = _max_decompress_length()
+ decompressor = bz2.BZ2Decompressor()
+ uncompressed = decompressor.decompress(data, limit + 1)
+ if len(uncompressed) > limit:
+ _raise_decompression_too_large(limit)
return avro.io.BinaryDecoder(io.BytesIO(uncompressed))
Review Comment:
BZip2Codec.decompress now uses bz2.BZ2Decompressor().decompress() once and
returns without verifying that the compressed stream reached EOF (or handling
concatenated bzip2 streams). This can silently accept truncated/corrupt blocks
(and can truncate output if buffered data remains), whereas bz2.decompress()
would error on invalid/truncated input and will process concatenated streams.
Consider looping until EOF, draining buffered output with empty input, and
processing any unused_data as additional streams; raise if input ends before
EOF.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
