agrawaldevesh commented on a change in pull request #29032:
URL: https://github.com/apache/spark/pull/29032#discussion_r457770003
##########
File path: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
##########
@@ -871,7 +871,7 @@ private[deploy] class Master(
logInfo("Telling app of decommission executors")
exec.application.driver.send(ExecutorUpdated(
exec.id, ExecutorState.DECOMMISSIONED,
- Some("worker decommissioned"), None, workerLost = false))
+ Some("worker decommissioned"), None, workerLost = true))
Review comment:
On second thoughts, this is a bigger change: I will just explain why
workerLost = true better in the code.
The main problem is that the Deploy messages are b/w the Master and the
Driver, while the ExecutorDecommissionInfo are b/w the Driver and the
Executors. In any case, each cluster manager would need to have its own way of
somehow communicating the decommissioning metadata to the driver, and this
workerLost could be the Master's internal way.
The alternatives would be to either introduce another
`ExecutorDecommissioned(id, MasterToDriverExecutorDecommissionInfo)` struct, or
enhance `ExecutorUpdated.workerLost` to be a full Struct/Any that can take the
MasterToDriverExecutorDecommissionInfo.
This all seems like a bit too much machinery for now, so I will just comment
it inline in the code.
Thank you for raising this question :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
