Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/5663#issuecomment-96741575
> So what does YARN show the status of the application as in this case
(when run in yarn-client mode)?
Hi @tgravescs , I think that's actually out of the scope of this change.
The backend here is fine; when the net is shaky, connections will be torn down
and executors will go away since they lost connection to the driver. Retrying
to connect to the RM on the driver side won't help fix that (although you could
potentially recover at some point and then have new executors allocated for
you).
What I understand from the bug report is that when that happens, the driver
process will stay around and not leave for some reason. Looking at the code I
don't fully understand how that happens (seeing a `jstack` output for the
process would maybe clear that out), but that's what the change is trying to
fix here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
