[
https://issues.apache.org/jira/browse/SPARK-19380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Zhang updated SPARK-19380:
------------------------------
Description:
SPARK-13723 only uses user's number of executors as the initial number of
executors when dynamic allocation is turned on.
If the configured max number of executors is larger than the number of
executors requested by the user, user's application could continue to request
for more executors to reach the configured max number if there're tasks backed
up. This behavior is not very friendly to the cluster if we allow every Spark
application to reach the max number of executors.
> YARN - Dynamic allocation should use configured number of executors as max
> number of executors
> ----------------------------------------------------------------------------------------------
>
> Key: SPARK-19380
> URL: https://issues.apache.org/jira/browse/SPARK-19380
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 1.6.3
> Reporter: Zhe Zhang
>
> SPARK-13723 only uses user's number of executors as the initial number of
> executors when dynamic allocation is turned on.
> If the configured max number of executors is larger than the number of
> executors requested by the user, user's application could continue to request
> for more executors to reach the configured max number if there're tasks
> backed up. This behavior is not very friendly to the cluster if we allow
> every Spark application to reach the max number of executors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
