Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20217
Yea, @gatorsmile's
https://github.com/apache/spark/pull/20217#issuecomment-357131129 this was
exactly what was on my mind.
I have few arguments to
https://github.com/apache/spark/pull/20217#issuecomment-357130083:
> I kind of like the PySpark API registerUDF. I think the API is simple and
clear, comparing to the alternative (merging this API to registerFunction and
throwing an error in case that f is a UDF and returnType is set)
It's cleaner and simple but historically I believe we are trying to not add
new functions and APIs so far. There is already a hell of a lot of APIs and
functions.
> I also don't think it's super important to be same as Scala API there
because Python doesn't have the same type system as Scala.
It's quite important convention we have tried to do so far. Language
specific sugar like Python's decorator, etc. can be supported differently but
I believe primarily exposed APIs are consistent.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
