Weijun-H commented on code in PR #23614:
URL: https://github.com/apache/datafusion/pull/23614#discussion_r3588820888
##########
datafusion/core/benches/sort.rs:
##########
@@ -813,9 +813,38 @@ where
.collect()
}
-type AxisGenerator = Box<dyn Fn(DataProfile, Cardinality, usize) ->
PartitionedBatches>;
+fn create_single_partition<T, F>(input: Vec<T>, f: F) -> Vec<RecordBatch>
+where
+ T: Clone,
+ F: Fn(Vec<T>) -> RecordBatch,
+{
+ input.chunks(BATCH_SIZE).map(|x| f(x.to_vec())).collect()
Review Comment:
nit: since this helper owns `input`, could it consume the vector into
batch-sized chunks instead of requiring `T: Clone`? This setup is outside the
measured iteration, so it does not affect benchmark timings, but avoiding the
copy would reduce setup time and peak memory across the full `sort_axis` matrix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
