Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/3009#issuecomment-63371456
> The only other issue is something we discussed earlier, which is what
happens if a map stage has to be recomputed due to a fetch failure? Do you end
up with more complete stages than stages in the job?
I'd like to add a Selenium test for this. It looks like I should be able
to simulate a fetch failure by throwing `FetchFailedException` from a `map()`
task. Do you know if there's an easy way to get a shuffleId and mapId from
inside of a `map()` UDF? If not, is there an easier way to simulate fetch
failures for UI testing?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
