Saisai Shao created SPARK-14788:
-----------------------------------
Summary: Initial number of executors should honor min number if
streaming dynamic allocation is enabled
Key: SPARK-14788
URL: https://issues.apache.org/jira/browse/SPARK-14788
Project: Spark
Issue Type: Bug
Components: Streaming
Affects Versions: 2.0.0
Reporter: Saisai Shao
Streaming dynamic allocation has lower bound
{{spark.streaming.dynamicAllocation.minExecutors}} and upper bound
{{spark.streaming.dynamicAllocation.maxExecutors}} of executor number. But
seems it is not used when starting on yarn or other cluster manager, I think we
should honor {{minExecutors}} as an initial number of executors for starting if
streaming dynamic allocation is enabled, like what we did in Spark dynamic
allocation.
>From my understanding this is an issue should be fixed, but I'm not sure it is
>a by-design choice, what is your opinion [~andrewor14] and [~tdas] ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
