cloud-fan commented on code in PR #40707:
URL: https://github.com/apache/spark/pull/40707#discussion_r1162266204
##########
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala:
##########
@@ -929,6 +929,13 @@ private[spark] class TaskSetManager(
info.id, taskSet.id, tid, ef.description))
return
}
+ if (ef.className == classOf[SparkUserException].getName) {
Review Comment:
One thing we should think about is how to differentiate user-facing error
and user-triggered error. We may still need to retry for user-facing error,
e.g. file read error which can be transient.
One idea is to have a special prefix for error classes that should still
trigger retry, such as file read error and OOM, which shouldn't be many.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
