pitrou commented on code in PR #51128:
URL: https://github.com/apache/arrow/pull/51128#discussion_r3912012359
##########
cpp/src/parquet/decoder.cc:
##########
@@ -1561,6 +1561,20 @@ class DeltaBitPackDecoder : public
TypedDecoderImpl<DType> {
}
total_values_remaining_ = total_value_count_;
+ // GH-50314: mini_blocks_per_block_ comes from the page header and sizes
the
+ // allocation below, while InitBlock() reads one bit-width byte per
miniblock.
+ // A count larger than the bytes left can never decode, so we reject it
here
+ // instead of allowing it to drive a large allocation. A page holding a
single
+ // value keeps that value in the header and never calls InitBlock(), so
this
Review Comment:
I think Copilot is right that we should also handle `total_value_count_ ==
1` somehow @1fanwang . Perhaps in that case we should just skip the allocation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
