ozankabak commented on code in PR #6956:
URL: https://github.com/apache/arrow-datafusion/pull/6956#discussion_r1265104516
##########
datafusion/core/src/physical_plan/coalesce_partitions.rs:
##########
@@ -107,6 +107,10 @@ impl ExecutionPlan for CoalescePartitionsExec {
self.input.equivalence_properties()
}
+ fn ordering_equivalence_properties(&self) -> OrderingEquivalenceProperties
{
Review Comment:
For posterity: There could be two meanings we can assign to ordering
equivalences:
1. Entries in the ordering equivalence set describe different, alternative
descriptions of *the current table ordering*. This is @mustafasrepo's
interpretation.
2. Entries in the ordering equivalence set describe different, alternative
sorts one can apply to this table and end up with the same ordering. This is
@alamb's interpretation.
(2) is more general than (1), and it opens up possibilities for more
optimizations. However, it is also more difficult to implement in all cases.
One can construct examples where the invariant described by (2) is lost when
one has order-destroying operators interspersed in between order-imposing
operators in deep-ish plans.
We just had a nice discussion with @mustafasrepo on this, and we added
generalizing from (1) to (2) to our roadmap of future improvements. Let's get
(1) done in a robust way first, and in the future we will revisit this and
figure out how to get to (2), and apply even more optimizations!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
