gene-bordegaray commented on code in PR #22590:
URL: https://github.com/apache/datafusion/pull/22590#discussion_r3321255183
##########
datafusion/physical-expr/src/partitioning.rs:
##########
@@ -442,6 +521,42 @@ impl Partitioning {
}
}
+ /// Returns true when `self` and `other` describe compatible partition
maps.
+ ///
+ /// Compatible partition maps can be used for partition-local behavior: if
+ /// this returns true, partition `i` from both partitionings can be treated
+ /// as covering the same partition domain. This is stricter than
+ /// [`Self::satisfaction`], which only answers whether this partitioning
can
+ /// satisfy a required distribution.
Review Comment:
like for example we might have two partitonings: `Hash(col a)` and
`Range(col a, split points:[10, 20, 30])`.
If a partitioned join where to require the distribution `Hash(col a)` but we
had `Range(col a, split points:[10, 20, 30])`. We could say that this satisfies
that partitioning since `Hash(col a)` requires each partiton to have a unique
set of `col a` values that no other partition has. But we can not say that
these two partitoning schemes are compatible because `Hash(col a)` and
`Range(col a, split points:[10, 20, 30])` will map rows differently to
partitions thus would break things, such as pushing a dynamic filter built from
hash partitioning into a `DataSourceExec` that is range partitioned (like we
saw in #21207).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
