Copilot commented on code in PR #4599:
URL: https://github.com/apache/datafusion-comet/pull/4599#discussion_r3361023778
##########
native/shuffle/src/partitioners/partitioned_batch_iterator.rs:
##########
@@ -85,18 +86,19 @@ impl<'a> PartitionedBatchIterator<'a> {
pos: 0,
}
}
-}
-
-impl Iterator for PartitionedBatchIterator<'_> {
- type Item = datafusion::common::Result<RecordBatch>;
- fn next(&mut self) -> Option<Self::Item> {
+ /// Returns the next shuffled batch, recording the gather cost into
`interleave_time`.
+ pub(crate) fn next(
+ &mut self,
+ interleave_time: &Time,
+ ) -> Option<datafusion::common::Result<RecordBatch>> {
if self.pos >= self.indices.len() {
return None;
}
let indices_end = std::cmp::min(self.pos + self.batch_size,
self.indices.len());
let indices = &self.indices[self.pos..indices_end];
+ let _timer = interleave_time.timer();
match interleave_record_batch(&self.record_batches, indices) {
Ok(batch) => {
Review Comment:
The `interleave_time` metric timer is currently stopped via drop (`let
_timer = ...`). In this crate, other `Time` measurements explicitly call
`timer.stop()` once the measured section finishes, which avoids including
error-wrapping / bookkeeping overhead in the metric and keeps timing style
consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
