Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/14162
> What is the case you are seeing this issue? I'm ok with changing it if we
have a good reason.
Well, I guess "good reason" is in the eye of the beholder. :-) My argument
for the change is to avoid users complaining when their app fails because of
this. Instead, they'd have a clear warning that something is not right on a
certain NM, and the rest of the world would keep going, using other NMs.
I can see the argument for not affecting other services; but how does the
MR service behave?
Also, the current behavior leads not only to app failures, but also to a
lot of noise in the NM's logs. So at least that part could be fixed, if people
really don't want the behavior change, so that the shuffle service is really
not running instead of running in a broken state.
Finally, this is a single error spot that is being filtered... e.g., if
there's a port conflict, the Spark shuffle service will throw an exception and
prevent the NM from starting.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
