Rafael Acevedo created CALCITE-6664:
---------------------------------------
Summary: GREATEST function null behavior not consistent with Spark
Key: CALCITE-6664
URL: https://issues.apache.org/jira/browse/CALCITE-6664
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.38.0
Reporter: Rafael Acevedo
Even though spark docs [1] indicates that GREATEST should skip null values.
Tests [2] indicate that it should not (any null params makes it return null).
As shown below, greatest(1, null) in Spark returns 1. In calcite, we only get
this behavior for Postgres.
{code:java}
❯ spark-sql
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
setLogLevel(newLevel).
24/11/01 08:01:13 WARN NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
24/11/01 08:01:16 WARN HiveConf: HiveConf of name hive.stats.jdbc.timeout does
not exist
24/11/01 08:01:16 WARN HiveConf: HiveConf of name hive.stats.retries.wait does
not exist
24/11/01 08:01:19 WARN ObjectStore: Version information not found in metastore.
hive.metastore.schema.verification is not enabled so recording the schema
version 2.3.0
24/11/01 08:01:19 WARN ObjectStore: setMetaStoreSchemaVersion called but
recording version is disabled: version = 2.3.0, comment = Set by MetaStore
rafael.acevedo@192.168.15.7
24/11/01 08:01:19 WARN ObjectStore: Failed to get database default, returning
NoSuchObjectException
Spark Web UI available at http://192.168.15.7:4040
Spark master: local[*], Application Id: local-1730458874925
spark-sql (default)> select greatest(1, null);
1
Time taken: 2.346 seconds, Fetched 1 row(s)
spark-sql (default)> {code}
Question is: is this a bug or intended behavior?
[1] [https://spark.apache.org/docs/latest/api/sql/index.html#greatest]
[2]
https://github.com/apache/calcite/blob/659ea980b077ed6721debc27f19eedaa9616e0ab/testkit/src/main/java/org/apache/calcite/test/SqlOperatorTest.java#L11440-L11441
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
