mingmwang commented on PR #5171: URL: https://github.com/apache/arrow-datafusion/pull/5171#issuecomment-1420174086
@mustafasrepo @ozankabak Regarding the rule applying ordering, since DataFusion optimization framework is still a traditional heuristic style framework, the rule applying orders always matter, we can not assume one rule can work independently without the others. Specifically , `EnforceDistribution` rule is responsible for handling the global distribution requirements. And `EnforceSorting` rule is responsible for handling the local sort requirements. It's also responsible for removing unnecessary global sort and local sort. The global distribution requirements need to be handled first, after that we can handle the local sort(inner-partition) requirements. Global properties vs Local properties http://www.cs.albany.edu/~jhh/courses/readings/zhou10.pdf -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
