Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/900#discussion_r13977443
--- Diff:
yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala
---
@@ -77,6 +77,12 @@ private[spark] class YarnClientSchedulerBackend(
logDebug("ClientArguments called with: " + argsArrayBuf)
val args = new ClientArguments(argsArrayBuf.toArray, conf)
+ totalExecutors.set(args.numExecutors)
+ // reset default minRegisteredRatio for yarn mode
+ if (minRegisteredRatio == 0) {
+ minRegisteredRatio = 0.9
--- End diff --
I'm a bit on the fence about making the default 90%. While I want it high
on yarn I was originally thinking this would just be set by a cluster config
(user level) and leave the default in Spark code as 0 for backwards
compatibility. So its probably better to remove this and leave default 0 unless
others disagree.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
