[
https://issues.apache.org/jira/browse/SPARK-21834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin resolved SPARK-21834.
------------------------------------
Resolution: Fixed
Assignee: Sital Kedia
Fix Version/s: 2.3.0
2.2.1
2.1.2
> Incorrect executor request in case of dynamic allocation
> --------------------------------------------------------
>
> Key: SPARK-21834
> URL: https://issues.apache.org/jira/browse/SPARK-21834
> Project: Spark
> Issue Type: Bug
> Components: Scheduler
> Affects Versions: 2.2.0
> Reporter: Sital Kedia
> Assignee: Sital Kedia
> Fix For: 2.1.2, 2.2.1, 2.3.0
>
>
> killExecutor api currently does not allow killing an executor without
> updating the total number of executors needed. In case of dynamic allocation
> is turned on and the allocator tries to kill an executor, the scheduler
> reduces the total number of executors needed ( see
> https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala#L635)
> which is incorrect because the allocator already takes care of setting the
> required number of executors itself.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
