mustafasrepo commented on code in PR #11875:
URL: https://github.com/apache/datafusion/pull/11875#discussion_r1711596337
##########
datafusion/core/src/physical_optimizer/enforce_distribution.rs:
##########
@@ -1031,6 +1031,75 @@ fn replace_order_preserving_variants(
context.update_plan_from_children()
}
+/// A struct to keep track of RepartitionRequirement status for each child
node.
+struct RepartitionRequirementStatus {
+ roundrobin_beneficial: bool,
+ hash_necessary: bool,
+}
+
+/// Calculates the `RepartitionRequirementStatus` for each children to
generate consistent requirements.
+/// As an example, for hash exec left children might produce
`RepartitionRequirementStatus{roundrobin_beneficial: true, hash_necessary:
true}`
+/// and right children might produce:
`RepartitionRequirementStatus{roundrobin_beneficial: false, hash_necessary:
false}`.
+/// When target partitions=4, left child might produce `Hash(vec![expr], 4)`
and right child might produce `Hash(vec![expr], 4)`. However,
+/// for correct operation we need consistent hashes accross children. This
util turns right child status:
+/// from `RepartitionRequirementStatus{roundrobin_beneficial: false,
hash_necessary: false}` into
+/// `RepartitionRequirementStatus{roundrobin_beneficial: false,
hash_necessary: true}` to generate consistent plan.
+fn get_repartition_status_flags(
Review Comment:
Changed it,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
