Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/9766#issuecomment-176371911
Oh right, I was thinking more not wrapping the UDF and turning it into a
Python UDF (thats going to kill performance) - but more if one has Scala/Java
UDFs is the overhead of registering it in Scala code and use it in Python (e.g.
writing a function like
https://github.com/sparklingpandas/sparklingpandas/blob/master/src/main/scala/com/sparklingpandas/AggregationUDFs.scala#L36
and then calling it with py4j) high enough having a wrapper to do it directly
from python is useful?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
