alamb commented on issue #11413:
URL: https://github.com/apache/datafusion/issues/11413#issuecomment-2232921403

   > Sorry it takes longer than I expected to make this works end-to-end. I 
plan to make an ScalarUDF with arrow-udf as the completion of the prototype 
work.
   
   Thank you so much @xinlifoobar  -- this is really helpful and a great 
analysis (I think the pros/cons you identified make a lot of sense to me)
   
   From what I can see, if we wanted to proceed with using `arrow-udf` in 
DataFusion we would need to address the shortcomings you identified above. 
   
   Here are some additional discussions
   
   > By default all udf are private, lack of a way to reference the udf that 
could be used in e.g., ExprPlanner.
   
   I think this is part of the same concept as discussed on  
https://lists.apache.org/thread/x8wvlkfr0osl15o52rw85wom0p4v05x6 -- basically 
the arrow-udf library's scope is large enough to encompass things like a 
function registry that DataFusion already has
   
   > Lack of support for operations against array and scalar.
   
   I do think being able to special case scalar value is a critical requirement 
for performance. 
   
   I will post about your findings on the mailing lists and let's see what the 
authors of arrow-udf have to say
   
   cc @wangrunji0408 @Xuanwo


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org

Reply via email to