Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/900#discussion_r13749934
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -48,6 +48,10 @@ class CoarseGrainedSchedulerBackend(scheduler:
TaskSchedulerImpl, actorSystem: A
var totalCoreCount = new AtomicInteger(0)
val conf = scheduler.sc.conf
private val timeout = AkkaUtils.askTimeout(conf)
+ val minRegisteredNum = conf.getDouble("spark.executor.minRegisteredNum",
0)
--- End diff --
I agree that the default should be zero to keep same behavior, but it
should also be able to be changed to a reasonable default for all people using
a cluster. For instance, I want all spark on yarn customers using grid x to
have this default to 90% of their executors available before starting because
that is what gives them a good experience. Like I said if that doesn't make
sense for other deployment modes or its very hard to implement then we can work
around for yarn, but its adding yet another config which I would prefer not to
do
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
