wangyum opened a new pull request #27874: [WIP][SPARK-31114][SQL] Constraints inferred from equality constraints with cast URL: https://github.com/apache/spark/pull/27874 ### What changes were proposed in this pull request? This PR add support inferred morre constraints from equality constraints with cast, For e.g., ```scala spark.sql("CREATE TABLE SPARK_31114_1(a BIGINT)") spark.sql("CREATE TABLE SPARK_31114_2(b DECIMAL(18, 0))") spark.sql("SELECT t1.* FROM SPARK_31114_1 t1 JOIN SPARK_31114_2 t2 ON t1.a = t2.b AND t1.a = 1L").explain ``` Before this PR: ``` == Physical Plan == *(2) Project [a#0L] +- *(2) BroadcastHashJoin [cast(a#0L as decimal(20,0))], [cast(b#1 as decimal(20,0))], Inner, BuildRight :- *(2) Project [a#0L] : +- *(2) Filter (isnotnull(a#0L) AND (a#0L = 1)) : +- *(2) ColumnarToRow : +- FileScan parquet default.spark_31114_1[a#0L] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, decimal(18,0), true] as decimal(20,0)))), [id=#50] +- *(1) Project [b#1] +- *(1) Filter isnotnull(b#1) +- *(1) ColumnarToRow +- FileScan parquet default.spark_31114_2[b#1] ``` After this PR: ``` *(2) Project [a#218L] +- *(2) BroadcastHashJoin [cast(a#218L as decimal(20,0))], [cast(b#219 as decimal(20,0))], Inner, BuildRight :- *(2) Project [a#218L] : +- *(2) Filter (isnotnull(a#218L) AND (a#218L = 1)) : +- *(2) ColumnarToRow : +- FileScan parquet default.spark_31114_1[a#218L] +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, decimal(18,0), true] as decimal(20,0)))), [id=#119] +- *(1) Project [b#219] +- *(1) Filter ((cast(b#219 as bigint) = 1) AND isnotnull(b#219)) +- *(1) ColumnarToRow +- FileScan parquet default.spark_31114_2[b#219] ``` ### Why are the changes needed? Improve query performance. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Unit test.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
