Yuming Wang created SPARK-34802: ----------------------------------- Summary: Optimize Optimizer defaultBatches rule order Key: SPARK-34802 URL: https://issues.apache.org/jira/browse/SPARK-34802 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 3.2.0 Reporter: Yuming Wang
For example: {code:scala} spark.sql("CREATE TABLE t1 using parquet AS SELECT id AS a FROM range(10)") spark.sql("CREATE TABLE t2 using parquet AS SELECT id AS b FROM range(10)") spark.sql("SELECT * FROM t1 INNER JOIN t2 ON (a = b AND true)").explain {code} {noformat} === Applying Rule org.apache.spark.sql.catalyst.optimizer.PushDownPredicates === !Join Inner, ((a#4L = b#5L) AND true) Join Inner, (a#4L = b#5L) !:- Relation default.t1[a#4L] parquet :- Filter true !+- Relation default.t2[b#5L] parquet : +- Relation default.t1[a#4L] parquet ! +- Relation default.t2[b#5L] parquet {noformat} BooleanSimplification should before PushDownPredicates. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org