mbutrovich commented on code in PR #329:
URL: https://github.com/apache/datafusion-spark/pull/329#discussion_r4605627544


##########
crates/spark/src/function/datetime/make_date.rs:
##########
@@ -0,0 +1,538 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: See the NOTICE file distributed with this work for 
additional information regarding copyright ownership
+// SPDX-FileContributor: Licensed to the Apache Software Foundation (ASF) 
under one or more contributor license agreements
+
+use std::sync::Arc;
+
+use arrow::array::{Array, Date32Array, Date32Builder, new_empty_array};
+use arrow::datatypes::{DataType, Field, FieldRef};
+use datafusion_common::cast::as_int32_array;
+use datafusion_common::types::{NativeType, logical_int32, logical_string};
+use datafusion_common::utils::take_function_args;
+use datafusion_common::{Result, ScalarValue, exec_err, internal_err};
+use datafusion_expr::{
+    Coercion, ColumnarValue, Documentation, ReturnFieldArgs, 
ScalarFunctionArgs,
+    ScalarUDFImpl, Signature, TypeSignatureClass, Volatility,
+};
+use datafusion_macros::user_doc;
+
+const ARITHMETIC_OVERFLOW_ERROR: &str = r#"[ARITHMETIC_OVERFLOW] overflow. If 
necessary set "spark.sql.ansi.enabled" to "false" to bypass this error. 
SQLSTATE: 22003"#;
+
+/// Spark-compatible `make_date` expression.
+///
+/// <https://spark.apache.org/docs/latest/api/sql/index.html#make_date>
+#[user_doc(
+    doc_section(label = "Time and Date Functions"),
+    description = "Creates a date from `year`, `month` and `day` fields. 
DataFusion has no built-in \
+        equivalent. Years from -999999999 to 999999999 are accepted (year 0 is 
1 BC), as long as \
+        the date fits the DATE range. When the fields do not form a valid date 
(for example \
+        month 13 or February 29 in a non-leap year), the function returns 
NULL, or raises \
+        DATETIME_FIELD_OUT_OF_BOUNDS when 
`datafusion.execution.enable_ansi_mode` is true. A \
+        valid date outside the DATE range (before -5877641-06-23 or after 
+5881580-07-11) raises \
+        ARITHMETIC_OVERFLOW in both modes, as in Spark. Other numeric and 
string inputs are cast \
+        to INT first, which truncates fractions and raises an error if the 
value is out of the \
+        INT range or is not a valid number. These errors match Spark with ANSI 
mode enabled. \
+        With ANSI mode disabled, Spark wraps an out-of-range BIGINT and 
returns NULL for a \
+        malformed string, while this function still errors (see \

Review Comment:
   > Added in 777724a as commented-out queries citing #88, and the sentence now 
covers NaN, infinite DOUBLEs and out-of-range DECIMALs.
   
   I got the DECIMAL case wrong in my first comment, sorry. With ANSI mode off, 
Spark casts to INT with `numeric.toInt` 
([`Cast.scala`](https://github.com/apache/spark/blob/32f7299601108917fb01920a54e084595b7b3bf8/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala#L945-L947)),
 which for a DECIMAL is `toLong.toInt` 
([`Decimal.scala`](https://github.com/apache/spark/blob/32f7299601108917fb01920a54e084595b7b3bf8/sql/api/src/main/scala/org/apache/spark/sql/types/Decimal.scala#L259)).
 So an out-of-range DECIMAL wraps like a BIGINT. My `99999999999` example 
returns NULL only because it wraps to `1215752191`, which is past the year 
range. On Spark 4.2.0 with ANSI mode off, `make_date(CAST(4294969320 AS 
DECIMAL(20,0)), 1, 1)` returns `2024-01-01`. Here it raises `Cast error: value 
of 4294969320 is out of range Int32`.
   
   A DOUBLE out of the INT range saturates instead. `CAST(CAST(1e10 AS DOUBLE) 
AS INT)` and `CAST(CAST('Infinity' AS DOUBLE) AS INT)` are both `2147483647`, 
which is never a valid field. So `make_date` returns NULL for any of them, not 
only for infinities.
   
   Could the sentence in the description ([`make_date.rs` 
L35-L37](https://github.com/apache/datafusion-spark/blob/0f6d284f39459e5d2e250b093f8f7e2246f57d6c/crates/spark/src/function/datetime/make_date.rs#L35-L37))
 read like this?
   
   ```
   With ANSI mode disabled, Spark wraps an out-of-range BIGINT or DECIMAL, 
returns NULL for a
   malformed string or a floating-point value out of the INT range, and casts 
NaN to 0, while
   this function still errors
   ```
   
   Could the comment above the commented-out queries in `make_date.slt` 
([L125-L127](https://github.com/apache/datafusion-spark/blob/0f6d284f39459e5d2e250b093f8f7e2246f57d6c/crates/sqllogictest/test_files/datetime/make_date.slt?plain=1#L125-L127))
 say the same, with the wrapping case added?
   
   ```
   # query D
   # SELECT make_date(CAST(4294969320 AS DECIMAL(20,0)), 1, 1);
   # ----
   # 2024-01-01
   ```
   
   The `int32_coercion()` comment ([`date_part_name.rs` 
L39-L40](https://github.com/apache/datafusion-spark/blob/0f6d284f39459e5d2e250b093f8f7e2246f57d6c/crates/spark/src/function/datetime/date_part_name.rs#L39-L40))
 says Spark "returns NULL or wraps". For `date_from_unix_date` the saturation 
gives a date: `date_from_unix_date(CAST(1e10 AS DOUBLE))` is `+5881580-07-11` 
on Spark 4.2.0 with ANSI mode off. Could that comment say "returns NULL, wraps, 
or saturates"?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to