andygrove opened a new issue, #315:
URL: https://github.com/apache/datafusion-spark/issues/315

   ### Describe the bug
   
   `make_dt_interval` can return NULL for non-NULL inputs, but 
`return_field_from_args` 
([datetime/make_dt_interval.rs#L146](https://github.com/apache/datafusion-spark/blob/21ab12776f55d5646431d81f76b55f6bd0b3ea1e/crates/spark/src/function/datetime/make_dt_interval.rs#L146))
 declares the result non-nullable when every argument is non-nullable and the 
seconds are finite. When the expression is not constant-folded, the batch fails 
validation.
   
   ### To Reproduce
   
   Run with the `simplify_expressions` optimizer rule removed 
(`Optimizer::new().rules` without it, passed to 
`SessionStateBuilder::with_optimizer_rules`):
   
   ```sql
   SELECT make_dt_interval(1500000, 0, 0, 0);
   -- Arrow error: Invalid argument error: Column 
'make_dt_interval(Int64(1500000),Int64(0),Int64(0),Int64(0))'
   -- is declared as non-nullable but contains null values
   ```
   
   With the default optimizer the call is constant-folded to a NULL literal, so 
the query returns NULL.
   
   ### Expected behavior
   
   The result field is nullable whenever the function can return NULL for 
non-NULL input. Today that is any overflow, and also the range in #109. Once 
#109 and #20 are fixed (Spark returns the interval or raises an error), the 
result may be non-nullable again, so the nullability should follow whatever 
those fixes do.
   
   ### Additional context
   
   Found by the new `evaluate` mode of the sqllogictest runner, which runs the 
`.slt` files without `simplify_expressions`. That query is skipped in 
`evaluate` mode until this is fixed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to