coderfender commented on code in PR #357:
URL: https://github.com/apache/datafusion-spark/pull/357#discussion_r4621357688


##########
crates/sqllogictest/test_files/array/array_repeat.slt:
##########
@@ -105,3 +105,63 @@ query ?
 select array_repeat('a', NULL);
 ----
 NULL
+
+# Spark's `ArrayRepeat` takes an INT count. An integer count of another type is
+# cast to INT, because DataFusion types `2` as BIGINT.
+query ?
+SELECT array_repeat('a', 2::TINYINT);
+----
+[a, a]
+
+query ?
+SELECT array_repeat('a', 2::SMALLINT);
+----
+[a, a]
+
+query ?
+SELECT array_repeat('a', 2::BIGINT);
+----
+[a, a]
+
+query ?
+SELECT array_repeat('a', arrow_cast(2, 'UInt32'));
+----
+[a, a]
+
+query ?
+SELECT array_repeat(1, c) FROM (VALUES (2), (NULL), (0)) AS t(c);
+----
+[1, 1]
+NULL
+[]
+
+# A BIGINT count outside the INT range fails the cast to INT.
+query error Cast error

Review Comment:
   Optional: Spark rejects this one at analysis with `UNEXPECTED_INPUT_TYPE` 
(the literal is BIGINT), rather than a cast error. Since `coerce_types` only 
sees types, the cast error makes sense here; a short note in the comment above 
might help future readers.



##########
crates/spark/src/function/array/repeat.rs:
##########
@@ -89,29 +90,28 @@ impl ScalarUDFImpl for SparkArrayRepeat {
     fn coerce_types(&self, arg_types: &[DataType]) -> Result<Vec<DataType>> {
         let [first_type, second_type] = take_function_args(self.name(), 
arg_types)?;
 
-        // Coerce the second argument to Int64/UInt64 if it's a numeric type
-        let second = match second_type {
-            DataType::Int8
-            | DataType::Int16
-            | DataType::Int32
-            | DataType::Int64
-            | DataType::Null => DataType::Int64,
-            DataType::UInt8 | DataType::UInt16 | DataType::UInt32 | 
DataType::UInt64 => {
-                DataType::UInt64
-            }
-            _ => return exec_err!("count must be an integer type"),
-        };
-
-        Ok(vec![first_type.clone(), second])
+        // Spark's `ArrayRepeat` takes an INT count. DataFusion types bare 
integer
+        // literals as BIGINT, so accept every integer type and cast to INT. A
+        // value outside the INT range then fails the cast instead of being
+        // silently truncated.
+        if !(second_type.is_integer() || second_type.is_null()) {

Review Comment:
   Non-blocking nit: a dictionary-encoded integer count isn't unwrapped here, 
since `is_integer()` is false for e.g. `Dictionary(Int32, Int64)`. Because 
`spark_sql_type` unwraps dictionaries, the error then reads `requires the "INT" 
type, however it has the type "BIGINT"`, which is a bit confusing given a plain 
BIGINT is accepted. It was rejected before this PR too, so not a regression. 
Matching on the dictionary's value type would line it up with the convention 
from #349.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to