Github user squito commented on the pull request:
https://github.com/apache/spark/pull/8760#issuecomment-165280794
@mridulm going back to one of your previous comments:
> What would be more ideal is an implementation which allows for
blacklisting single task (as is current), to multiple tasks for a taskset on a
single executor (in case the taskset has issues on the executor - memory
constraints perhaps) to blacklisting the node : time bound ofcourse, and for
specific taskset and/or all taskset until timeout expires (directory cleanup
catching up, etc).
Did I miss this in there somewhere ?
this isn't specifically included, but I think it could be added as an
alternate strategy without a huge change. If I understand correctly, you're
basically proposing adding another blacklisted state, where an executor is
completely blacklisted, but only for *one taskset*, and after its blacklisted
for multiple tasksets then its promoted to being blacklisted for all tasksets
(still subject to some timeout or something)? I don't think that would be too
hard but also trying to find the balance between adding too many knobs -- is
there a clear use case for it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
