Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/19048
I think I'm starting to understand what you're getting at, but I still
don't see why this has anything to do with the CGSB. What I understand from
your comment is that the EAM may reduce its target and at the same time try to
kill idle executors, basically doubling down and killing too many executors in
the process.
Isn't this what this piece of code is trying to prevent?
```
} else if (newExecutorTotal - 1 < numExecutorsTarget) {
logDebug(s"Not removing idle executor $executorIdToBeRemoved
because there are only " +
s"$newExecutorTotal executor(s) left (number of executor target
$numExecutorsTarget)")
```
If killing an idle executor would bring the number below the current
target, and it won't be killed. That's a pretty recent fix so maybe you haven't
seen it (SPARK-21656).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
