andygrove commented on code in PR #3752:
URL: https://github.com/apache/datafusion-comet/pull/3752#discussion_r3024304942
##########
native/shuffle/src/metrics.rs:
##########
@@ -33,6 +33,15 @@ pub(crate) struct ShufflePartitionerMetrics {
/// Time spent writing to disk. Maps to "shuffleWriteTime" in Spark SQL
Metrics.
pub(crate) write_time: Time,
+ /// Time spent in interleave_record_batch gathering rows into shuffled
batches
+ pub(crate) interleave_time: Time,
+
+ /// Time spent coalescing small batches before serialization
+ pub(crate) coalesce_time: Time,
+
+ /// Time spent buffering partition indices and memory accounting
+ pub(crate) memcopy_time: Time,
Review Comment:
I have now removed this new metric so that this PR is just for adding the
benchmark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
