alamb commented on issue #12819: URL: https://github.com/apache/datafusion/issues/12819#issuecomment-2444190036
> It would be great to figure out a way to pass this nullability info (or return) the invoke at execution time. Do you think any of the suggestion in the initial post would be a good idea? I worry about making too many different combinations of invoke / invoke_no_args / invoke_nullable, etc > > It seems like the nullability wasn't fully considered for function since the `is_nullable` and `return_type_from_exprs` where added later, would be great have this info throughout the trait. I agree nullability was not fully considered and having a proper nullability story would be good. I think one thing that happened is that much of DataFusion's code assumes the input can be nullable and then switches to special case no-null code at runtime based on [`Array::null_count`](https://docs.rs/arrow/latest/arrow/array/trait.Array.html#method.null_count) rather than relying on the schema That made ensuring that the schema's nullability aligned wasn't as important and so there are a bunch of places which are fast and loose with it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
