Ran Tao created CALCITE-5893:
--------------------------------
Summary: Wrong NULL literal behavior of
ARRAY_CONTAINS/ARRAY_EXCEPT/ARRAY_INTERSECT In Spark Library
Key: CALCITE-5893
URL: https://issues.apache.org/jira/browse/CALCITE-5893
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0
Reporter: Ran Tao
Assignee: Ran Tao
Fix For: 1.36.0
The following are the implementations of some functions in calcite(Spark
Library) that are inconsistent with spark.
The reason here is that *null* and *cast (null as xxx)* are treated equally,
and *NullPolicy* acts on these two situations at the same time and returns null
directly. However in spark, the former needs to throw an exception, and the
correct form is the latter. (In fact, apache flink also throws an exception.)
we should use Resource.nullIllegal to raise "Illegal use of ''NULL'" in such
case to match Spark behavior.
*calcite spark:*
// return null
select array_contains(array[1, 2], null);
// return null
select array_except(array[1, 2, 3], null)
// return null
select array_intersect(array[1,2,3], null)
*actual spark:*
{code:java}
// Cannot resolve "array_contains(array(1, 2), NULL)" due to data type
mismatch: // Null typed values cannot be used as arguments of `array_contains`
spark-sql (default)> select array_contains(array(1, 2), null); {code}
{code:java}
// data type mismatch: Input to function `array_except` should have been two
"ARRAY" with same element type,
// but it's ["ARRAY<INT>", "VOID"]
spark-sql (default)> select array_except(array(1, 2, 3), null); {code}
{code:java}
// data type mismatch: Input to function `array_intersect` should have been two
"ARRAY" with same element type,
// but it's ["ARRAY<INT>", "VOID"]
spark-sql (default)> select array_intersect(array(1,2,3), null); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
