Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/17113
@tgravescs , the main scenario is external shuffle service unavailable
scenario, this could be happened in working preserving + NM failure situation.
Also like Mesos + external standalone shuffle service could introduce this
issue. In scenarios like rolling upgrade I agreed that NM unavailability is
short and this issue could be self-recoverable. One scenario I'm simulating is
NM failure. In my test, when NM is failed, RM will detect this failure after 10
minutes by default, before that executors on that NM can still serve the tasks,
and Spark doesn't blacklist these containers, so re-issued tasks could still be
failed.
`FetchFailed` will immediately abort the running stage and re-issue parent
stage, configurations like failed task number per stage may not be so useful,
so my thinking is to backlist these executors/nodes immediately after fetch
failure.
This proposal may have many problems for different scenario, that's why I
opened here for comments. If you don't think it is necessary to fix then I
could close it.
@markhamstra this patch is targeted to master branch and all the
investigations and changes is based on master branch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
