tgravescs commented on a change in pull request #28257:
URL: https://github.com/apache/spark/pull/28257#discussion_r414124538
##########
File path:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
##########
@@ -741,8 +750,12 @@ private[spark] class TaskSchedulerImpl(
if (state == TaskState.LOST) {
// TaskState.LOST is only used by the deprecated Mesos
fine-grained scheduling mode,
// where each executor corresponds to a single task, so mark the
executor as failed.
- val execId = taskIdToExecutorId.getOrElse(tid, throw new
IllegalStateException(
- "taskIdToTaskSetManager.contains(tid) <=>
taskIdToExecutorId.contains(tid)"))
+ val execId = taskIdToExecutorId.getOrElse(tid, {
+ val errorMsg =
+ "taskIdToTaskSetManager.contains(tid) <=>
taskIdToExecutorId.contains(tid)"
+ taskSet.abort(errorMsg)
+ throw new SparkException(errorMsg)
Review comment:
yea that should be fine, as it will just be caught and then logged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
