[
https://issues.apache.org/jira/browse/ARROW-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284933#comment-17284933
]
Hendrik Makait commented on ARROW-11625:
----------------------------------------
Moving this check into the constructor leads to failing tests. As far as I can
see, this is because the planner does an optimization step that inserts a merge
for children that contain multiple partitions.
{code:java}
match plan.required_child_distribution() {
Distribution::UnspecifiedDistribution => plan.with_new_children(children),
Distribution::SinglePartition => plan.with_new_children(
children
.iter()
.map(|child| {
if child.output_partitioning().partition_count() == 1 {
child.clone()
} else {
Arc::new(MergeExec::new(child.clone()))
}
})
.collect(),
),
}
{code}
What's the reason for moving this check into planning time? How should I
proceed?
> [Rust] [DataFusion] Move SortExec partition check to constructor
> ----------------------------------------------------------------
>
> Key: ARROW-11625
> URL: https://issues.apache.org/jira/browse/ARROW-11625
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Rust - DataFusion
> Reporter: Andy Grove
> Priority: Major
> Fix For: 4.0.0
>
>
> SortExec has the following error check at execution time and this could be
> moved into the try_new constructor so the error check happens at planning
> time instead.
>
> {code:java}
> if 1 != self.input.output_partitioning().partition_count() {
> return Err(DataFusionError::Internal(
> "SortExec requires a single input partition".to_owned(),
> ));
> } {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
