aokolnychyi commented on code in PR #40707:
URL: https://github.com/apache/spark/pull/40707#discussion_r1161947582
##########
core/src/main/resources/error/error-classes.json:
##########
@@ -5569,5 +5569,15 @@
"message" : [
"Failed to get block <blockId>, which is not a shuffle block"
]
+ },
+ "_LEGACY_ERROR_TEMP_3043" : {
+ "message" : [
+ "User exception: <msg>"
+ ]
+ },
+ "_LEGACY_ERROR_TEMP_3044" : {
Review Comment:
Shall we define a proper error class? There is
`NOT_NULL_CONSTRAINT_VIOLATION` that seems relevant. It is currently used to
validate NOT NULL constraints for array elements and map values.
##########
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala:
##########
@@ -929,6 +929,13 @@ private[spark] class TaskSetManager(
info.id, taskSet.id, tid, ef.description))
return
}
+ if (ef.className == classOf[SparkUserException].getName) {
Review Comment:
I wonder whether this logic can be generic and apply to all exceptions that
extend `SparkThrowable` and have an error class defined?
cc @cloud-fan @gengliangwang @HyukjinKwon
##########
core/src/main/scala/org/apache/spark/SparkException.scala:
##########
@@ -355,3 +355,24 @@ private[spark] class SparkSQLFeatureNotSupportedException(
override def getErrorClass: String = errorClass
}
+
+/**
+ * User error exception thrown from Spark with an error class.
+ */
+private[spark] class SparkUserException(
Review Comment:
Question: Are we sure a custom exception is needed for this case? Is there
any existing exceptions we can reuse with NPE as cause?
If we want to have a brand new exception, what about
`SparkNotNullConstraintViolationException` to be more specific? I guess it will
depend whether we want to skip retries only for this exception type as opposed
to all Spark exceptions with known error codes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
