yashaggarwal85d commented on code in PR #26637:
URL: https://github.com/apache/datafusion/pull/26637#discussion_r4366069067
##########
datafusion/physical-plan/src/aggregates/clustered_single_stream.rs:
##########
@@ -851,4 +872,70 @@ mod tests {
);
Ok(())
}
+
+ /// Runs a single `SUM(k) GROUP BY k` over two batches sorted on `k` that
+ /// share a boundary key. Returns the output and the most memory the stream
+ /// held while handing off output.
+ async fn run_fully_sorted(
Review Comment:
Agreed. I moved it into one shared `#[cfg(test)] mod clustered_test_util`
next to the stream modules. It provides `sum_over_sorted_input(mode, keys,
batches)`, which builds the plan (for `Final`, it builds the partial-state
input), and `collect_with_held_memory(&Arc<dyn ExecutionPlan>, pool)`, which
runs it through `execute` with `enable_migration_aggregate` on and temp files
off. I put it in its own file instead of inside `aggregates/mod.rs` because
that file is already very large; happy to move it if you prefer.
`clustered_partial_stream.rs` now uses it too. I didn't add the SLT case: as
you noted, it would depend on exact byte sizes and could be fragile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
