Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/7786#issuecomment-142122398
@jerryshao could you rebase the PR and address the feedback?
One thing that would be nice is to actually take action on the driver to
get rid of executors that are in the "strict" set if they are idle. That can
help YARN in freeing resources earlier, and could also allow for a replacement
executor on a non-contended node to be started earlier, since the preemption
cycle can take some time. These executors could be in this "preempted" state
for a long time (several seconds), which is enough to run a lot of small tasks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
