Jefffrey commented on code in PR #10262:
URL: https://github.com/apache/arrow-rs/pull/10262#discussion_r3511242741
##########
arrow-ipc/src/writer.rs:
##########
@@ -615,14 +615,16 @@ impl IpcDataGenerator {
) -> Result<(Vec<EncodedData>, EncodedData), ArrowError> {
let encoded_dictionaries =
self.encode_all_dicts(batch, dictionary_tracker, write_options,
ipc_write_context)?;
- let mut arrow_data = Vec::new();
+ let mut arrow_data = std::mem::take(&mut ipc_write_context.scratch);
let (ipc_message, _, tail_pad) = self.record_batch_to_bytes(
batch,
write_options,
ipc_write_context,
&mut IpcBodySink::Write(&mut arrow_data),
)?;
arrow_data.extend_from_slice(&PADDING[..tail_pad]);
+ let final_capcity = arrow_data.capacity();
+ ipc_write_context.scratch.reserve(final_capcity); // reset scratch to
the same capacity as before, due to
['FlightDataEncoder::split_batch_for_grpc_response'] we know that batches are
split up into roughly equal sized chunks,
Review Comment:
nit but i do wonder does this mean the final encode call will always
allocate again but not use scratch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
