Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/159#issuecomment-37774958
In case of FetchFailed, it does not help to blacklist the executor (since
failure was not local to that executor).
TaskKilled is something I am unsure of - either is fine I guess. I am not
sure in what all circumstances it will get fired; and if any of them can be due
to running the task in node issues. Any thoughts ?
Note that when the number of executors are reasonably high, blacklisting
some for a task would be ok - but when number of executors is low and number of
tasks per stage is not too high (or towards end of a stage), aggressively
marking executors as failed can slow the stage down a lot from our observation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
