Github user e-dorigatti commented on the issue:
https://github.com/apache/spark/pull/21383
Yes, the problem was that the signature is lost when the function is
wrapped, and the worker needs the signature to know whether the function needs
keys together with values or not.
What I meant is that fixing worker side might fix this specific JIRA, but
the bug will still occur when an udf is not used in a worker, but somewhere
else. If you fix at the udf side, however, the bug will never occur again,
regardless of how and where the udf is used. I don't know the codebase well
enough to know whether this will be a real problem, though
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
