Github user nishkamravi2 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7532#discussion_r35961635
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -804,6 +826,85 @@ private[master] class Master(
}
/**
+ * Handle a request to set the target number of executors for this
application.
+ *
+ * If the executor limit is adjusted upwards, new executors will be
launched provided
+ * that there are workers with sufficient resources. If it is adjusted
downwards, however,
+ * we do not kill existing executors until we explicitly receive a kill
request.
+ *
+ * @return whether the application has previously registered with this
Master.
+ */
+ private def handleRequestExecutors(appId: String, requestedTotal: Int):
Boolean = {
+ idToApp.get(appId) match {
+ case Some(appInfo) =>
+ logInfo(s"Application $appId requested to set total executors to
$requestedTotal.")
+ appInfo.executorLimit = requestedTotal
+ schedule()
+ true
+ case None =>
+ logWarning(s"Unknown application $appId requested $requestedTotal
total executors.")
+ false
+ }
+ }
+
+ /**
+ * Handle a kill request from the given application.
+ *
+ * This method assumes the executor limit has already been adjusted
downwards through
+ * a separate [[RequestExecutors]] message, such that we do not
immediately launch new
+ * executors immediately after the old ones are removed.
--- End diff --
Nit: redundant 'immediately'
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
