Swinky commented on a change in pull request #31318:
URL: https://github.com/apache/spark/pull/31318#discussion_r579372991
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
##########
@@ -366,6 +363,16 @@ object BooleanSimplification extends Rule[LogicalPlan]
with PredicateHelper {
// ((c || ...) && (d || ...)) || a || b
(common :+ And(ldiff.reduce(Or), rdiff.reduce(Or))).reduce(Or)
}
+ } else {
+ // No common factors from disjunctive predicates, reduce common
factor from conjunction
+ val all = splitConjunctivePredicates(left) ++
splitConjunctivePredicates(right)
Review comment:
Currently, `and` of common disjunctive predicates is optimized Example:
`(a || b || c || ...) && (a || b) => (a || b)`
but `and` of common conjunctive predicates is not optimized. Example: `((a
&& b) && (a && c))) => a && b && c`
The improvement is to optimize for common conjunctive predicate which can
have mix of `&` and `|`
Similarly for 'Or' case of common disjunctive predicate is added in this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
