Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21467
So .. @e-dorigatti, mind if I ask to elaborate and describe the current
approach within `fail_on_stopiteration`? Seems we will handle UDFs in worker
side and RDD APIs (which takes a function processing each record) will be when
the function is wrapped.
Let's try to explain this in details everywhere in the codes since it's
going to take a while for other guys (at least me) to understand. I think I
still slightly more prefer the current way to get rid of weird hacks. We
already have some divergence in SQL UDF vs RDD UDF.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
