ozankabak commented on code in PR #8169:
URL: https://github.com/apache/arrow-datafusion/pull/8169#discussion_r1392506421
##########
datafusion/physical-expr/src/equivalence.rs:
##########
@@ -575,10 +588,19 @@ impl OrderingEquivalenceClass {
}
}
- /// Gets the first ordering entry in this ordering equivalence class.
- /// This is one of the many valid orderings (if there are multiple).
+ /// Gets the concatenated version of the all orderings.
+ /// if orderings are [a ASC, b ASC] and [c ASC, d ASC]
+ /// Returns [a ASC, b ASC, c ASC, d ASC].
+ /// This ensures that during during merging
+ /// [a ASC, b ASC] and [c ASC, d ASC] are still valid.
pub fn output_ordering(&self) -> Option<LexOrdering> {
- self.orderings.first().cloned()
+ let output_ordering = self
+ .orderings
+ .iter()
+ .flat_map(|ordering| ordering.to_vec())
+ .collect::<Vec<_>>();
Review Comment:
Nit: We can probably write this more idiomatically and avoid collecting
individual pieces.
##########
datafusion/physical-expr/src/equivalence.rs:
##########
@@ -575,10 +588,19 @@ impl OrderingEquivalenceClass {
}
}
- /// Gets the first ordering entry in this ordering equivalence class.
- /// This is one of the many valid orderings (if there are multiple).
+ /// Gets the concatenated version of the all orderings.
+ /// if orderings are [a ASC, b ASC] and [c ASC, d ASC]
+ /// Returns [a ASC, b ASC, c ASC, d ASC].
+ /// This ensures that during during merging
+ /// [a ASC, b ASC] and [c ASC, d ASC] are still valid.
Review Comment:
```suggestion
/// Returns the concatenation of all the orderings. This enables merge
/// operations to preserve all equivalent orderings simultaneously.
```
##########
datafusion/physical-plan/src/repartition/mod.rs:
##########
@@ -472,9 +472,6 @@ impl ExecutionPlan for RepartitionExec {
if !self.maintains_input_order()[0] {
result.clear_orderings();
}
- if self.preserve_order {
- result =
result.with_reorder(self.sort_exprs().unwrap_or_default().to_vec())
- }
Review Comment:
For other reviewers: This is the key change that avoids losing alternative
orderings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
