[
https://issues.apache.org/jira/browse/CALCITE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Nuyanzin closed CALCITE-5893.
------------------------------------
Resolved in release 1.37.0 (2024-05-06)
> Wrong NULL operand behavior of ARRAY_CONTAINS/ARRAY_EXCEPT/ARRAY_INTERSECT In
> Spark Library
> -------------------------------------------------------------------------------------------
>
> Key: CALCITE-5893
> URL: https://issues.apache.org/jira/browse/CALCITE-5893
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.35.0
> Reporter: Ran Tao
> Assignee: Ran Tao
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.37.0
>
>
> The following are the implementations of some array functions in
> calcite({*}Spark Library{*}) that are inconsistent with actual spark behavior.
>
> The reason here is that *null* and *cast (null as xxx)* are treated equally,
> and *NullPolicy* acts on these two situations at the same time and returns
> null directly. However in spark, the former needs to throw an exception, and
> the correct behavior is the latter. (In fact, apache flink also throws an
> exception.) we should throw exception in such case to match Spark behavior.
> *calcite spark:*
> // return null
> select array_contains(array[1, 2], null);
> // return null
> select array_except(array[1, 2, 3], null)
> // return null
> select array_intersect(array[1,2,3], null)
> *actual spark:*
> {code:java}
> // Cannot resolve "array_contains(array(1, 2), NULL)" due to data type
> mismatch: // Null typed values cannot be used as arguments of `array_contains`
> spark-sql (default)> select array_contains(array(1, 2), null); {code}
> {code:java}
> // data type mismatch: Input to function `array_except` should have been two
> "ARRAY" with same element type,
> // but it's ["ARRAY<INT>", "VOID"]
> spark-sql (default)> select array_except(array(1, 2, 3), null); {code}
> {code:java}
> // data type mismatch: Input to function `array_intersect` should have been
> two "ARRAY" with same element type,
> // but it's ["ARRAY<INT>", "VOID"]
> spark-sql (default)> select array_intersect(array(1,2,3), null); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
