alamb commented on code in PR #11423:
URL: https://github.com/apache/arrow-rs/pull/11423#discussion_r4221031109
##########
parquet/src/arrow/array_reader/byte_array_dictionary.rs:
##########
@@ -221,6 +224,13 @@ where
}
fn read_records(&mut self, batch_size: usize) -> Result<usize> {
+ // Reclaim the previous batch's keys allocation if the consumer
dropped the output array; otherwise fall through and allocate fresh.
Review Comment:
this is an interesting idea -- and I think it could be more widely used
However, in this case I wonder if this is over fit for the benchmarks (like
how often will this actually be able to be reused 🤔
On the other hand, it probably does work for DataFusion 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
