ozankabak commented on code in PR #5322:
URL: https://github.com/apache/arrow-datafusion/pull/5322#discussion_r1114945924
##########
datafusion/core/src/execution/context.rs:
##########
@@ -1528,6 +1528,9 @@ impl SessionState {
// repartitioning and local sorting steps to meet distribution and
ordering requirements.
// Therefore, it should run before EnforceDistribution and
EnforceSorting.
Arc::new(JoinSelection::new()),
+ // Enforce sort before PipelineFixer
Review Comment:
We haven't focused on optimal rule ordering yet. Right now, we only enable
SHJ when joining two unbounded streams, so it has to happen at some point
before `PipelineFixer`. As we mature the SHJ implementation, we will enable it
even for normal tables when it is appropriate (yields performance gains).
Within that context, we will revisit rule ordering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
