Yuming Wang created SPARK-34802:
-----------------------------------
Summary: Optimize Optimizer defaultBatches rule order
Key: SPARK-34802
URL: https://issues.apache.org/jira/browse/SPARK-34802
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.2.0
Reporter: Yuming Wang
For example:
{code:scala}
spark.sql("CREATE TABLE t1 using parquet AS SELECT id AS a FROM range(10)")
spark.sql("CREATE TABLE t2 using parquet AS SELECT id AS b FROM range(10)")
spark.sql("SELECT * FROM t1 INNER JOIN t2 ON (a = b AND true)").explain
{code}
{noformat}
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.PushDownPredicates ===
!Join Inner, ((a#4L = b#5L) AND true) Join Inner, (a#4L = b#5L)
!:- Relation default.t1[a#4L] parquet :- Filter true
!+- Relation default.t2[b#5L] parquet : +- Relation default.t1[a#4L] parquet
! +- Relation default.t2[b#5L] parquet
{noformat}
BooleanSimplification should before PushDownPredicates.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
