Github user markgrover commented on a diff in the pull request:
https://github.com/apache/spark/pull/8007#discussion_r37093692
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -239,10 +240,10 @@ class CoarseGrainedSchedulerBackend(scheduler:
TaskSchedulerImpl, val rpcEnv: Rp
}
totalCoreCount.addAndGet(-executorInfo.totalCores)
totalRegisteredExecutors.addAndGet(-1)
- scheduler.executorLost(executorId, SlaveLost(reason))
+ scheduler.executorLost(executorId, reason)
listenerBus.post(
- SparkListenerExecutorRemoved(System.currentTimeMillis(),
executorId, reason))
- case None => logInfo(s"Asked to remove non-existent executor
$executorId")
+ SparkListenerExecutorRemoved(System.currentTimeMillis(),
executorId, reason.toString))
+ case None => logError(s"Asked to remove non-existent executor
$executorId")
--- End diff --
I'd love to hear your thoughts on why you decided to make this error? It
seems to me in the regular cycle of things, you'd still get this quite often -
once on onDisconnected and other on RemoveExecutor event, right? Would making
this an error give people the wrong impression?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
