2010YOUY01 commented on code in PR #15302:
URL: https://github.com/apache/datafusion/pull/15302#discussion_r2006777564
##########
datafusion/physical-plan/src/sorts/sort.rs:
##########
@@ -688,15 +707,29 @@ impl ExternalSorter {
let fetch = self.fetch;
let expressions: LexOrdering = self.expr.iter().cloned().collect();
- let stream = futures::stream::once(futures::future::lazy(move |_| {
- let timer = metrics.elapsed_compute().timer();
- let sorted = sort_batch(&batch, &expressions, fetch)?;
- timer.done();
+ let row_converter = Arc::clone(&self.sort_keys_row_converter);
+ let stream = futures::stream::once(async move {
+ let _timer = metrics.elapsed_compute().timer();
+
+ let sort_columns = expressions
+ .iter()
+ .map(|expr| expr.evaluate_to_sort_column(&batch))
+ .collect::<Result<Vec<_>>>()?;
+
+ let sorted = if is_multi_column_with_lists(&sort_columns) {
Review Comment:
Good point, now always using row format shows a mixed benchmark result,
because converted rows are not reused during sorting and sort-preserving
merging, and the conversion overhead outweighs the benefits.
We'd better do it as a follow-up with more benchmark analysis.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
