allisonwang-db commented on code in PR #43111:
URL: https://github.com/apache/spark/pull/43111#discussion_r1342014577
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala:
##########
@@ -63,6 +64,18 @@ object RewritePredicateSubquery extends Rule[LogicalPlan]
with PredicateHelper {
Join(outerPlan, dedupSubplan, joinType, condition, JoinHint(None, subHint))
}
+ private def removeDomainJoins(
Review Comment:
nit: `maybeRewriteDomainJoins`
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -3416,6 +3416,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val DECORRELATE_EXISTS_AND_IN_SUBQUERIES =
+ buildConf("spark.sql.optimizer.decorrelateExistsIn.enabled")
Review Comment:
Does this flag depend on this `DECORRELATE_INNER_QUERY_ENABLED` flag?
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -3416,6 +3416,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val DECORRELATE_EXISTS_AND_IN_SUBQUERIES =
+ buildConf("spark.sql.optimizer.decorrelateExistsIn.enabled")
+ .internal()
+ .doc("Decorrelate EXISTS and IN subqueries.")
+ .version("4.0.0")
+ .booleanConf
+ .createWithDefault(true)
Review Comment:
Will this have any query result changes if we enable this by default?
##########
sql/core/src/test/resources/sql-tests/inputs/subquery/exists-subquery/exists-count-bug.sql:
##########
@@ -0,0 +1,21 @@
+create temporary view t1(c1, c2) as values (0, 1), (1, 2);
+create temporary view t2(c1, c2) as values (0, 2), (0, 3);
+create temporary view t3(c1, c2) as values (0, 3), (1, 4), (2, 5);
+
+select * from t1 where exists (select count(*) from t2 where t2.c1 = t1.c1);
Review Comment:
are results before and after this PR the same for these queries?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
