Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3962#issuecomment-71910271
> so i think AMActor is unnecessary to listening to driver's disassociated
event in yarn cluster mode because userThread in AM can monitor any exceptions
of driver's main class. but AMActor donot know any exceptions of driver. Do you
understand what i said?
Yes I understand. Note however we already catch exceptions in the user
thread and `finish` with failed exit code
[here](https://github.com/apache/spark/blob/c8e934ef3cd06f02f9a2946e96a1a52293c22490/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala#L450).
Since `finish` can only be called once, when an exception occurs we already
exit with a non-zero code so the disassociation event shouldn't do anything.
However, this assumes that the disassociation event comes *after* the exception
is caught, which may not necessarily be true. For this reason I think it's
safer to leave the check in there as you suggest. We definitely need some
comment to explain why the behavior is different in cluster mode though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
