iemejia opened a new pull request, #582: URL: https://github.com/apache/avro-rs/pull/582
## What `Codec::decompress` grew the output buffer without any limit, so a small compressed block could inflate to an enormous buffer and exhaust memory (a "decompression bomb"): - **deflate** used `miniz_oxide::inflate::decompress_to_vec`, which grows the output `Vec` unbounded; - **snappy** allocated `vec![0; decompressed_size]` where `decompressed_size` comes from the (untrusted) block header; - **zstd / bzip2 / xz** decompressed the whole stream into a `Vec` via `io::copy` / `read_to_end`. A block byte buffer is capped at the allocation budget (`safe_len` in the container reader), but its *decompressed* output was not — so a ~few-KB block could expand to many GB. ## How Bound every codec's decompressed output by the existing configurable `max_allocation_bytes` budget: - **deflate:** use `decompress_to_vec_with_limit(stream, max_bytes)` and map the resulting `HasMoreOutput` status to a `MemoryAllocation` error. - **snappy:** validate the header-declared `decompressed_size` with `safe_len` before allocating `vec![0; …]`. - **zstd / bzip2 / xz:** read through `Read::take(max_bytes + 1)` and reject when the output exceeds the budget (memory stays bounded to `max_bytes + 1`). The limit stays configurable through `max_allocation_bytes`, consistent with the existing byte-length guards, and legitimate blocks within the budget still round-trip unchanged. ## Tests - New integration test `avro/tests/decompression_bomb.rs` (its own process so the write-once allocation limit can be lowered): for **deflate, snappy, zstandard, bzip2 and xz**, an 8 MiB-plaintext bomb that compresses to a few KB is rejected on decompress, while a small payload still round-trips. - Existing codec round-trip tests still pass; `cargo fmt --check` and `cargo clippy --tests` (all features) are clean. ## Notes This is a companion to #581 (bounding array/map element allocation); together they cap the two remaining unbounded-allocation paths in the decoder. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
