jorisvandenbossche commented on code in PR #38784:
URL: https://github.com/apache/arrow/pull/38784#discussion_r1404328273
##########
cpp/src/parquet/encoding.cc:
##########
@@ -1983,7 +2001,10 @@ class DictByteArrayDecoderImpl : public
DictDecoderImpl<ByteArrayType>,
int values_decoded = 0;
ArrowBinaryHelper<ByteArrayType> helper(out, num_values);
- RETURN_NOT_OK(helper.Prepare(len_));
+ // The `len_` in the ByteArrayDictDecoder is the total length of the
+ // RLE/Bit-pack encoded data size, so, we cannot use `len_` to reserve
+ // space for binary data.
+ // RETURN_NOT_OK(helper.Prepare());
auto dict_values = reinterpret_cast<const ByteArray*>(dictionary_->data());
Review Comment:
A little bit below, there is a `PrepareNextInput(val.len)` (current line
1999, new line 2020), which is showing up in a profile of reading the binary
column.
(now, you also already updated the implementation of `PrepareNextInput`, so
that might already have addressed it on this branch. Will run my profile with
this branch as well)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
