Niranjan Jayakar created SPARK-46074:
----------------------------------------
Summary: [CONNECT][SCALA] Insufficient details in error when a UDF
fails
Key: SPARK-46074
URL: https://issues.apache.org/jira/browse/SPARK-46074
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: 3.5.0
Reporter: Niranjan Jayakar
Currently, when a UDF fails the connect client does not receive the actual
error that caused the failure.
As an example, the error message looks like -
{code:java}
Exception in thread "main" org.apache.spark.SparkException:
grpc_shaded.io.grpc.StatusRuntimeException: INTERNAL: Job aborted due to stage
failure: Task 2 in stage 0.0 failed 4 times, most recent failure: Lost task 2.3
in stage 0.0 (TID 10) (10.68.141.158 executor 0):
org.apache.spark.SparkException: [FAILED_EXECUTE_UDF] Failed to execute user
defined function (` (Main$$$Lambda$4770/1714264622)`: (int) => int). SQLSTATE:
39000 {code}
In this case, the actual error was a {{{}java.lang.NoClassDefFoundError{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
