hvanhovell opened a new pull request, #42245: URL: https://github.com/apache/spark/pull/42245
### What changes were proposed in this pull request? This PR adds a better error message when a JVM UDF cannot be deserialized. ### Why are the changes needed? In some cases a UDF cannot be deserialized. The happens when a lambda references itself (typically through the capturing class). Java cannot deserialize such an object graph because SerializedLambda's are serialization proxies which need the full graph to be deserialized before they can be transformed into the actual lambda. This is not possible if there is such a cycle. This PR adds a more readable and understandable error when this happens, the original java one is a `ClassCastException` which does not explain or give any hints of what is actually going on. ### Does this PR introduce _any_ user-facing change? Yes. It will throw an error on the client when a UDF is not deserializable. The error is better and more actionable then what we got before. ### How was this patch tested? Added tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
