jonahgao commented on code in PR #12965:
URL: https://github.com/apache/datafusion/pull/12965#discussion_r1804008922
##########
datafusion/physical-expr/src/scalar_function.rs:
##########
@@ -147,8 +149,19 @@ impl PhysicalExpr for ScalarFunctionExpr {
if let ColumnarValue::Array(array) = &output {
if array.len() != batch.num_rows() {
- return internal_err!("UDF returned a different number of rows
than expected. Expected: {}, Got: {}",
- batch.num_rows(), array.len());
+ // If the function is not volatile and all arguments are
scalars,
+ // we can assume that returning a one-element array is
equivalent to returning a scalar.
+ let preserve_scalar = array.len() == 1
+ && self.fun.signature().volatility != Volatility::Volatile
+ && inputs
+ .iter()
+ .all(|arg| matches!(arg, ColumnarValue::Scalar(_)));
Review Comment:
I think we should consider the case where the inputs are empty. For UDFs
without args, either return a scalar directly or return an array with
`num_rows`. Trying to convert the output array back to scalar for them seems
unnecessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
