Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/3541#issuecomment-65272551
Thx @kayousterhout for the ping !
We are fairly aggressively using blacklisting executors - not hosts.
The assumption that a task failed on an executor in a host will imply it
will fail on all executors on the host is incorrect - particularly in
multi-tennet environment : where number of executors per host can be fairly
high : which is exactly the reason why I added this hack to begin with.
Instead of modifying the existing behavior of executor level blacklist, it
would be better to add to it - add a way to configure host level blacklisting
in addition to existing functionality.
I am -1 on changing the existing behavior.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
