[
https://issues.apache.org/jira/browse/SPARK-13002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Armbrust updated SPARK-13002:
-------------------------------------
Target Version/s: 2.0.0 (was: 1.6.1, 2.0.0)
> Mesos scheduler backend does not follow the property
> spark.dynamicAllocation.initialExecutors
> ---------------------------------------------------------------------------------------------
>
> Key: SPARK-13002
> URL: https://issues.apache.org/jira/browse/SPARK-13002
> Project: Spark
> Issue Type: Bug
> Components: Mesos
> Affects Versions: 1.5.2, 1.6.0
> Reporter: Luc Bourlier
> Labels: dynamic_allocation, mesos
>
> When starting a Spark job on a Mesos cluster, all available cores are
> reserved (up to {{spark.cores.max}}), creating one executor per Mesos node,
> and as many executors as needed.
> This is the case even when dynamic allocation is enabled.
> When dynamic allocation is enabled, the number of executor launched at
> startup should be limited to the value of
> {{spark.dynamicAllocation.initialExecutors}}.
> The Mesos scheduler backend already follows the value computed by the
> {{ExecutorAllocationManager}} for the number of executors that should be up
> and running. Expect at startup, when it just creates all the executors it can.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
