Github user squito commented on the issue:
https://github.com/apache/spark/pull/17297
> when the stage fails because of fetch failure, we remove the stage from
the output commiter. So if any task completes between the time of first fetch
failure and the time stage is resubmitted, will be denied to commit the output
oh, that is a great point. I was mostly thinking of another
shufflemapstage, where that wouldn't matter, but if its a result stage which
needs to commit its output, you are right.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
