[
https://issues.apache.org/jira/browse/SPARK-21933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-21933:
------------------------------------
Assignee: (was: Apache Spark)
> Spark Streaming request more executors than excepted without DynamicAllocation
> ------------------------------------------------------------------------------
>
> Key: SPARK-21933
> URL: https://issues.apache.org/jira/browse/SPARK-21933
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 2.2.0
> Reporter: Congxian Qiu
>
> When Spark Streaming application runs on Yarn without DynamicAllocation, If
> some nodemanager becomes lost, then the containers on the lost nodemanager
> will be reported to all the applicationmaster, application master will
> allocate new containers.
> But after application master allocate new containers, the lost nodemanager be
> available, after this, resource manager restarted, after resource manager has
> been restarted, the node manager will report all the containers running on
> it before to resource manager because of Yarn's HA, then application manager
> will receive a duplicated completed container message, so the spark streaming
> application will request more resource than it requires.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
