ozankabak commented on code in PR #7671:
URL: https://github.com/apache/arrow-datafusion/pull/7671#discussion_r1341636365
##########
datafusion/core/src/physical_optimizer/enforce_distribution.rs:
##########
@@ -3541,6 +3595,12 @@ mod tests {
];
assert_optimized!(expected, plan.clone(), true);
+
+ let expected = &[
Review Comment:
This plan is generated when the `bounded_order_preserving_variants`
configuration flag is `false`. When this flag is `true`, we get the sort-free
result. The prior behavior was basically ignoring/overriding the flag.
For some detailed context: The flag basically lets the user choose whether
they want `SortPreservingMerge`s, or `Repartition`/`Coalesce`+`Sort` cascades.
We ran some benchmarks and there is no clearly dominating strategy, each
alternative comes out ahead in certain cases. In non-streaming cases, the first
alternative typically came out ahead, so we let the default flag value to be
`false`.
Since we are a stream-first platform, we set the flag to `true` at Synnada.
Maybe IOx also wants to set this flag to `true`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
