Swinky Mann created SPARK-34598:
-----------------------------------
Summary: RewritePredicateSubquery Rule must not update Filters
without subqueries
Key: SPARK-34598
URL: https://issues.apache.org/jira/browse/SPARK-34598
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.1.1
Reporter: Swinky Mann
1. Currently RewritePredicateSubquery rule updates Filter node for queries
without any subquery as well. This shouldn't happen.
2. Also `Filter(conditions.reduce(And), child)` in the rule might create a
skewed expression tree even though original expression is balanced.
{noformat}
=== Applying Rule
org.apache.spark.sql.catalyst.optimizer.RewritePredicateSubquery ===
Project [a#0] Project
[a#0]
!+- Filter (((a#0 > 1) OR (b#1 > 2)) AND ((c#2 > 1) AND (d#3 > 2))) +- Filter
((((a#0 > 1) OR (b#1 > 2)) AND (c#2 > 1)) AND (d#3 > 2))
+- LocalRelation <empty>, [a#0, b#1, c#2, d#3] +-
LocalRelation <empty>, [a#0, b#1, c#2, d#3]{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
