GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/20604
[WIP][SPARK-23365][CORE] Do not adjust num executors when killing idle
executors.
The ExecutorAllocationManager should not adjust the target number of
executors when killing idle executors, as it has already adjusted the
target number down based on the task backlog.
The change is more than just flipping the value of `replace` because
`replace` also implied failure handling. Furthermore, the name
`replace` was misleading with DynamicAllocation on, as the target number
of executors is changed outside of the call to `killExecutors`.
TODO:
testing
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark SPARK-23365
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20604.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20604
----
commit b5a39dab324be4bb358682720cf0f7e55272559d
Author: Imran Rashid <irashid@...>
Date: 2018-02-13T22:07:26Z
[SPARK-23365] Do not adjust num executors when killing idle executors.
The ExecutorAllocationManager should not adjust the target number of
executors when killing idle executors, as it has already adjusted the
target number down based on the task backlog.
The change is more than just flipping the value of `replace` because
`replace` also implied failure handling. Furthermore, the name
`replace` was misleading with DynamicAllocation on, as the target number
of executors is changed outside of the call to `killExecutors`.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
