agrawaldevesh commented on a change in pull request #29015:
URL: https://github.com/apache/spark/pull/29015#discussion_r453843780
##########
File path: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
##########
@@ -863,7 +872,36 @@ private[deploy] class Master(
true
}
- private def decommissionWorker(worker: WorkerInfo): Unit = {
+ private def handleDecommissionWorkers(hostPorts: Seq[String]): Integer = {
+ val hostsWithoutPorts = new HashSet[String]
+ val addressToRemove = new HashSet[RpcAddress]
+ hostPorts.foreach(hp => {
Review comment:
The challenge here is that Its not a "clean filter" due to the presence
of hosts without ports. The main use case is actually to decommission a given
set of hosts (because the external system may not have visibility into the
worker ports). Perhaps I can split this into a union of sorts ... let me see.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
