[
https://issues.apache.org/jira/browse/SPARK-21834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411897#comment-16411897
]
Imran Rashid commented on SPARK-21834:
--------------------------------------
SPARK-23365 is basically a duplicate of this, though they both have changes
associated with them (though I didn't realize it at the time, SPARK-23365 is
not strictly necessary on top of this, but does improve code clarity).
> Incorrect executor request in case of dynamic allocation
> --------------------------------------------------------
>
> Key: SPARK-21834
> URL: https://issues.apache.org/jira/browse/SPARK-21834
> Project: Spark
> Issue Type: Bug
> Components: Scheduler
> Affects Versions: 2.2.0
> Reporter: Sital Kedia
> Assignee: Sital Kedia
> Priority: Major
> Fix For: 2.1.2, 2.2.1, 2.3.0
>
>
> killExecutor api currently does not allow killing an executor without
> updating the total number of executors needed. In case of dynamic allocation
> is turned on and the allocator tries to kill an executor, the scheduler
> reduces the total number of executors needed ( see
> https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala#L635)
> which is incorrect because the allocator already takes care of setting the
> required number of executors itself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
