[
https://issues.apache.org/jira/browse/SPARK-43491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KuijianLiu updated SPARK-43491:
-------------------------------
Attachment: image-2023-05-13-13-14-55-853.png
> In expression not compatible with EqualTo Expression
> ----------------------------------------------------
>
> Key: SPARK-43491
> URL: https://issues.apache.org/jira/browse/SPARK-43491
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.1
> Reporter: KuijianLiu
> Priority: Minor
> Attachments: image-2023-05-13-13-14-55-853.png
>
>
> The query results of Spark SQL 3.1.1 and Hive SQL 3.1.0 are inconsistent
> with same sql. Spark SQL calculates `{{{}0 in ('00')`{}}} as false, which act
> different from `{{{}=`{}}} keyword, but Hive calculates true. Hive is
> compatible with the `{{{}in`{}}} keyword in 3.1.0, but SparkSQL does not.
> It's better when dataTypes of elements in `{{{}In`{}}} expression are the
> same, it should behaviour as same as BinaryComparison like ` {{{}EqualTo`{}}}.
> Test SQL:
> {code:java}
> scala> spark.sql("select 1 as test where 0 = '00'").show
> +----+
> |test|
> +----+
> | 1|
> +----+
> scala> spark.sql("select 1 as test where 0 in ('00')").show
> +----+
> |test|
> +----+
> +----+
> scala> spark.sql("select 1 as test where 0 = '00'").explain(true)
> == Parsed Logical Plan ==
> 'Project [1 AS test#23]
> +- 'Filter (0 = 00)
> +- OneRowRelation== Analyzed Logical Plan ==
> test: int
> Project [1 AS test#23]
> +- Filter (0 = cast(00 as int))
> +- OneRowRelation== Optimized Logical Plan ==
> Project [1 AS test#23]
> +- OneRowRelation== Physical Plan ==
> *(1) Project [1 AS test#23]
> +- *(1) Scan OneRowRelation[]
> scala> spark.sql("select 1 as test where 0 in ('00')").explain(true)
> == Parsed Logical Plan ==
> 'Project [1 AS test#25]
> +- 'Filter 0 IN (00)
> +- OneRowRelation== Analyzed Logical Plan ==
> test: int
> Project [1 AS test#25]
> +- Filter cast(0 as string) IN (cast(00 as string))
> +- OneRowRelation== Optimized Logical Plan ==
> LocalRelation <empty>, [test#25]== Physical Plan ==
> LocalTableScan <empty>, [test#25]
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
