Jefffrey opened a new issue, #19982: URL: https://github.com/apache/datafusion/issues/19982
### Describe the bug https://github.com/apache/datafusion/blob/e5e76366a6200dcd7b85cc030128a191e3763dd1/datafusion/expr/src/udf.rs#L401-L419 The `ScalarValue`s in `scalar_arguments` don't necessarily have types which match `arg_fields` which can lead to confusion and extra handling in UDF definitions. ### To Reproduce ```rust // datafusion/core/tests/user_defined/user_defined_scalar_functions.rs #[tokio::test] async fn test_scalar_arg_types() -> Result<()> { #[derive(Debug, PartialEq, Eq, Hash)] struct TestUdf { signature: Signature, } impl Default for TestUdf { fn default() -> Self { Self { signature: Signature::coercible( vec![Coercion::new_implicit( TypeSignatureClass::Native(logical_int16()), vec![TypeSignatureClass::Numeric], NativeType::Int16, )], Volatility::Immutable, ), } } } impl ScalarUDFImpl for TestUdf { fn as_any(&self) -> &dyn Any { self } fn name(&self) -> &str { "test_udf" } fn signature(&self) -> &Signature { &self.signature } fn return_type(&self, _arg_types: &[DataType]) -> Result<DataType> { unreachable!() } fn return_field_from_args(&self, args: ReturnFieldArgs) -> Result<FieldRef> { assert_eq!(args.arg_fields.len(), 1); assert_eq!(args.scalar_arguments.len(), 1); assert_eq!( args.arg_fields[0].data_type(), &args.scalar_arguments[0].unwrap().data_type() ); // <-- This assert failing Ok( Field::new(self.name(), args.arg_fields[0].data_type().clone(), true) .into(), ) } fn invoke_with_args(&self, args: ScalarFunctionArgs) -> Result<ColumnarValue> { assert!(matches!( args.args[0], ColumnarValue::Scalar(ScalarValue::Int16(Some(_))) )); Ok(args.args[0].clone()) } } let ctx = SessionContext::new(); ctx.register_udf(TestUdf::default().into()); ctx.sql("select test_udf(1)").await?.collect().await?; Ok(()) } ``` Running this test: ```sh datafusion (main)$ cargo test -p datafusion --test user_defined_integration user_defined::user_defined_scalar_functions::test_scalar_arg_types -- --nocapture --exact Finished `test` profile [unoptimized + debuginfo] target(s) in 0.38s Running tests/user_defined_integration.rs (/Users/jeffrey/.cargo_target_cache/debug/deps/user_defined_integration-53cd66eeac0051d3) running 1 test thread 'user_defined::user_defined_scalar_functions::test_scalar_arg_types' (14600918) panicked at datafusion/core/tests/user_defined/user_defined_scalar_functions.rs:2264:13: assertion `left == right` failed left: Int16 right: Int64 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace test user_defined::user_defined_scalar_functions::test_scalar_arg_types ... FAILED failures: failures: user_defined::user_defined_scalar_functions::test_scalar_arg_types test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 79 filtered out; finished in 0.01s error: test failed, to rerun pass `-p datafusion --test user_defined_integration` ``` The field says the argument should be an i16 according to signature, but the scalar argument is a i64. ### Expected behavior Either somehow get type coercion to apply on `scalar_arguments` (not sure if this is possible), or update documentation to make this limitation clear. ### Additional context Related code https://github.com/apache/datafusion/blob/e5e76366a6200dcd7b85cc030128a191e3763dd1/datafusion/expr/src/expr_schema.rs#L576-L614 Might also affect udaf/udwf? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
