jayzhan211 commented on code in PR #15822:
URL: https://github.com/apache/datafusion/pull/15822#discussion_r2057125845
##########
datafusion/physical-optimizer/src/enforce_sorting/replace_with_order_preserving_variants.rs:
##########
@@ -137,6 +137,12 @@ fn plan_with_order_preserving_variants(
return Ok(sort_input);
} else if is_coalesce_partitions(&sort_input.plan) && is_spm_better {
let child = &sort_input.children[0].plan;
+ let mut fetch = fetch;
+ // Also need to check if the coalesce node has a fetch
+ if let Some(coalesce_fetch) = sort_input.plan.fetch() {
+ // Get the min fetch between the `fetch` and the coalesce's fetch:
+ fetch = Some(coalesce_fetch.min(fetch.unwrap_or(usize::MAX)))
Review Comment:
If they expect 10 rows because they have `limit 10` but we return 5 rows for
them, I think this is not ideal. return error seems like a better choice
##########
datafusion/physical-optimizer/src/enforce_sorting/replace_with_order_preserving_variants.rs:
##########
@@ -137,6 +137,12 @@ fn plan_with_order_preserving_variants(
return Ok(sort_input);
} else if is_coalesce_partitions(&sort_input.plan) && is_spm_better {
let child = &sort_input.children[0].plan;
+ let mut fetch = fetch;
+ // Also need to check if the coalesce node has a fetch
+ if let Some(coalesce_fetch) = sort_input.plan.fetch() {
+ // Get the min fetch between the `fetch` and the coalesce's fetch:
+ fetch = Some(coalesce_fetch.min(fetch.unwrap_or(usize::MAX)))
Review Comment:
If they expect 10 rows because they have `limit 10` but we return 5 rows for
them, I think this is not ideal, return error seems like a better choice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
