wangyum commented on a change in pull request #31857:
URL: https://github.com/apache/spark/pull/31857#discussion_r597072815
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelation.scala
##########
@@ -30,6 +31,7 @@ import org.apache.spark.sql.catalyst.rules._
* - Join with one or two empty children (including Intersect/Except).
* 2. Unary-node Logical Plans
* - Project/Filter/Sample/Join/Limit/Repartition with all empty children.
+ * - Join with false condition.
Review comment:
Mainly used to avoid unnecessary pushdown:
```sql
SELECT * FROM t1 LEFT JOIN t2 ON true
```
Plan:
```
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.PushDownPredicates
===
Join LeftOuter, true Join LeftOuter, true
:- Relation default.t1[a#6L,b#7L] parquet :- Relation
default.t1[a#6L,b#7L] parquet
!+- Relation default.t2[x#8L,y#9L] parquet +- Filter true
! +- Relation
default.t2[x#8L,y#9L] parquet
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.PruneFilters ===
Join LeftOuter, true Join LeftOuter, true
:- Relation default.t1[a#6L,b#7L] parquet :- Relation
default.t1[a#6L,b#7L] parquet
!+- Filter true +- Relation
default.t2[x#8L,y#9L] parquet
! +- Relation default.t2[x#8L,y#9L] parquet
00:57:41.206 ERROR org.apache.spark.sql.catalyst.rules.PlanChangeLogger:
=== Result of Batch Cleanup filters that cannot be pushed down ===
Join LeftOuter, true Join LeftOuter, true
:- Relation default.t1[a#6L,b#7L] parquet :- Relation
default.t1[a#6L,b#7L] parquet
!+- Filter true +- Relation
default.t2[x#8L,y#9L] parquet
! +- Relation default.t2[x#8L,y#9L] parquet
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
