andygrove opened a new issue, #109: URL: https://github.com/apache/datafusion-spark/issues/109
### Describe the bug `make_dt_interval` returns NULL for values that fit in the result. `make_interval_dt_nano` ([make_dt_interval.rs#L185](https://github.com/apache/datafusion-spark/blob/main/crates/spark/src/function/datetime/make_dt_interval.rs#L185)) adds up hours and minutes in `i32`, so `days * 24 * 60` overflows once `days` is above 1,491,308. The result is a microsecond count in `i64`, which can hold about 106,751,991 days. Spark does this arithmetic in `Long`. ### To Reproduce ```sql SELECT make_dt_interval(1491308, 0, 0, 0); -- 1491308 days 0 hours 0 mins 0.000000 secs SELECT make_dt_interval(1500000, 0, 0, 0); -- datafusion-spark: NULL -- Spark 4.2.0: INTERVAL '1500000 00:00:00' DAY TO SECOND ``` Spark 4.2.0 also returns a value for `make_dt_interval(106751991, 0, 0, 0)`. ### Expected behavior Do the intermediate arithmetic in `i64`, so that every value that fits in the result is returned. ### Additional context This is separate from #20, which is about what happens on a real overflow (NULL here, an error in Spark). Spark results come from PySpark 4.2.0 with a UTC session. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
