Luc Bourlier created SPARK-13002:
------------------------------------
Summary: Mesos scheduler backend does not follow the property
spark.dynamicAllocation.initialExecutors
Key: SPARK-13002
URL: https://issues.apache.org/jira/browse/SPARK-13002
Project: Spark
Issue Type: Bug
Components: Mesos
Affects Versions: 1.6.0, 1.5.2
Reporter: Luc Bourlier
When starting a Spark job on a Mesos cluster, all available cores are reserved
(up to {{spark.cores.max}}), creating one executor per Mesos node, and as many
executors as needed.
This is the case even when dynamic allocation is enabled.
When dynamic allocation is enabled, the number of executor launched at startup
should be limited to the value of {{spark.dynamicAllocation.initialExecutors}}.
The Mesos scheduler backend already follows the value computed by the
{{ExecutorAllocationManager}} for the number of executors that should be up and
running. Expect at startup, when it just creates all the executors it can.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
