Copilot commented on code in PR #3854:
URL: https://github.com/apache/avro/pull/3854#discussion_r3567635982
##########
lang/c++/impl/DataFile.cc:
##########
@@ -627,12 +679,39 @@ void DataFileReaderBase::readDataBlock() {
const uint8_t *data;
size_t len;
+ size_t maxLength = maxDecompressLength();
+ // zlib tracks output in a uLong (z_stream::total_out), which is
only
+ // 32-bit on some platforms (e.g. Windows). Cap the working limit
to
+ // uLong's range so the buffer-size arithmetic below (which uses
+ // total_out) cannot wrap when a larger AVRO_MAX_DECOMPRESS_LENGTH
is
+ // configured.
+ const uLong zlibMax = std::numeric_limits<uLong>::max();
+ if (maxLength > zlibMax) {
+ maxLength = zlibMax;
+ }
while (ret != Z_STREAM_END && st->next(&data, &len)) {
zs.avail_in = static_cast<uInt>(len);
zs.next_in = const_cast<Bytef *>(data);
do {
if (zs.total_out == uncompressed.size()) {
- uncompressed.resize(uncompressed.size() +
zlibBufGrowSize);
+ // Reject a block that would decompress to more than
the
+ // allowed maximum, before growing the buffer further.
+ if (uncompressed.size() >= maxLength) {
+ (void) inflateEnd(&zs);
+ // At the trigger uncompressed.size() == maxLength
and
+ // inflate still has output, so the block is at
least
+ // maxLength + 1 bytes. Report that (like the
snappy/zstd
+ // errors) so the message is accurate and
consistent.
+ throw Exception(
+ "Decompressed block size {} exceeds the
maximum allowed of {} bytes",
+ uncompressed.size() + 1, maxLength);
Review Comment:
The over-limit deflate error reports `uncompressed.size() + 1`, which can
wrap when `maxLength` is `SIZE_MAX` (or when `maxLength` is clamped to `uLong`
max on 32-bit). That can yield a nonsensical size in the exception message
(e.g., 0), undermining the intent of the limit diagnostics. Compute the
would-be size with a saturating +1 to avoid overflow.
##########
lang/c++/impl/ZstdDecompressWrapper.cc:
##########
@@ -43,9 +43,21 @@ std::string ZstdDecompressWrapper::decompress(const
std::vector<char> &compresse
if (ZSTD_isError(ret)) {
throw Exception("ZSTD decompression error: {}",
ZSTD_getErrorName(ret));
}
+ // Reject before appending so the buffer never grows past the
limit.
+ if (out.pos > maxLength - uncompressed.size()) {
+ throw Exception(
+ "Decompressed block size {} exceeds the maximum allowed of
{} bytes",
+ uncompressed.size() + out.pos, maxLength);
+ }
Review Comment:
In the Zstd streaming path, the exception formats `uncompressed.size() +
out.pos`. If `maxLength` is configured near `SIZE_MAX`, that addition can
overflow and produce an incorrect (wrapped) size in the error message. Use a
saturating add when computing the reported size so the diagnostic remains
accurate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
