andygrove opened a new issue, #315: URL: https://github.com/apache/datafusion-spark/issues/315
### Describe the bug `make_dt_interval` can return NULL for non-NULL inputs, but `return_field_from_args` ([datetime/make_dt_interval.rs#L146](https://github.com/apache/datafusion-spark/blob/21ab12776f55d5646431d81f76b55f6bd0b3ea1e/crates/spark/src/function/datetime/make_dt_interval.rs#L146)) declares the result non-nullable when every argument is non-nullable and the seconds are finite. When the expression is not constant-folded, the batch fails validation. ### To Reproduce Run with the `simplify_expressions` optimizer rule removed (`Optimizer::new().rules` without it, passed to `SessionStateBuilder::with_optimizer_rules`): ```sql SELECT make_dt_interval(1500000, 0, 0, 0); -- Arrow error: Invalid argument error: Column 'make_dt_interval(Int64(1500000),Int64(0),Int64(0),Int64(0))' -- is declared as non-nullable but contains null values ``` With the default optimizer the call is constant-folded to a NULL literal, so the query returns NULL. ### Expected behavior The result field is nullable whenever the function can return NULL for non-NULL input. Today that is any overflow, and also the range in #109. Once #109 and #20 are fixed (Spark returns the interval or raises an error), the result may be non-nullable again, so the nullability should follow whatever those fixes do. ### Additional context Found by the new `evaluate` mode of the sqllogictest runner, which runs the `.slt` files without `simplify_expressions`. That query is skipped in `evaluate` mode until this is fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
