matthewmturner commented on pull request #1881: URL: https://github.com/apache/arrow-datafusion/pull/1881#issuecomment-1058415493
If i'm understanding everything correctly i think theres an important point worth making on how `datafusion` on its own can be used as a system, similar to spark. Specifically, the `datafusion-python` bindings i believe would fall in the category of system (@alamb i believe you had that view as well - please correct me if im wrong or if things have changed since then). on my side, im planning to use `datafusion-python` until size of data / etc require using ballista. with that in mind, my hope is that `datafusion-python` can have first class system like support - for example runtime registration of UDFs. I know there is existing support for UDF/UDAF in `datafusion-python` although it isnt clear to me how the changes here could impact that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
