Copilot commented on code in PR #50710:
URL: https://github.com/apache/arrow/pull/50710#discussion_r3678490948
##########
cpp/src/parquet/decoder.cc:
##########
@@ -2372,6 +2472,31 @@ class ByteStreamSplitDecoder<FLBAType> : public
ByteStreamSplitDecoderBase<FLBAT
}
};
+// Keep chunk rollover out of the hot decoder functions that instantiate this
helper.
+Status
+ArrowBinaryHelper<ByteArrayType,
::arrow::BinaryType>::AppendValueWithKnownSizeSlow(
+ const uint8_t* data, int32_t length, int64_t
estimated_remaining_data_length) {
+ RETURN_NOT_OK(PushChunk());
+ RETURN_NOT_OK(ReserveInitialChunkData(estimated_remaining_data_length));
+ chunk_space_remaining_ -= length;
+ --entries_remaining_;
+ builder_->UnsafeAppend(data, length);
Review Comment:
AppendValueWithKnownSizeSlow() assumes a new chunk can always fit `length`,
but it never re-checks `CanFit(length)` after PushChunk(). If a single value
exceeds `kBinaryMemoryLimit`, `chunk_space_remaining_ -= length` underflows and
`UnsafeAppend` bypasses builder validation, risking corrupted state / OOM. Add
an explicit size check and return an error when the value can’t fit in an empty
chunk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
