Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/20460#discussion_r165195110
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -267,7 +267,11 @@ private[deploy] class SparkSubmitArguments(args:
Seq[String], env: Map[String, S
&& Try(JavaUtils.byteStringAsBytes(executorMemory)).getOrElse(-1L)
<= 0) {
SparkSubmit.printErrorAndExit("Executor Memory cores must be a
positive number")
}
- if (executorCores != null && Try(executorCores.toInt).getOrElse(-1) <=
0) {
+ // Kubernetes mode allows fractional values for spark.executor.cores
so bypass this check in
+ // the Kubernetes mode.
+ if (!master.startsWith("k8s")
+ && executorCores != null
+ && Try(executorCores.toInt).getOrElse(-1) <= 0) {
--- End diff --
should this be nested `if..()` statements?
That's more readable IMO but not sure what's considered more idiomatic here.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
