Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/7532#discussion_r35509417
--- Diff:
core/src/main/scala/org/apache/spark/deploy/master/ApplicationInfo.scala ---
@@ -96,6 +109,47 @@ private[spark] class ApplicationInfo(
private[master] def coresLeft: Int = requestedCores - coresGranted
+ /**
+ * Return the number of executors waiting to be scheduled once space
frees up.
+ *
+ * This is only defined if the application explicitly set the executor
limit. For instance,
+ * if an application asks for 8 executors but there is only space for 5,
then there will be
+ * 3 waiting executors.
+ */
+ private[master] def numWaitingExecutors: Int = {
+ if (executorLimit != Integer.MAX_VALUE) {
+ math.max(0, executorLimit - executors.size)
+ } else {
+ 0
+ }
+ }
+
+ /**
+ * Add a worker to the blacklist, called when the executor running on
the worker is killed.
+ * This is used only if cores per executor is not set.
+ */
+ private[master] def blacklistWorker(workerId: String): Unit = {
+ blacklistedWorkers += workerId
+ }
+
+ /**
+ * Remove workers from the blacklist, called when the application
requests new executors.
+ * This is used only if cores per executor is not set.
+ */
+ private[master] def removeFromBlacklist(numWorkers: Int): Unit = {
+ blacklistedWorkers.take(numWorkers).foreach { workerId =>
--- End diff --
I removed the blacklist altogether. Never liked it anyway
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
