Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3962#issuecomment-71510021
@lianhuiwang Great that the latest changes are now much simpler and
minimal. However, I still don't fully agree with one point:
> so if AMActor subscribe to disassociated event and finish with
FinalApplicationStatus.SUCCEEDED, that's incorrect to do so
Why is it incorrect? Although the AM and the driver belong to the same
process, the driver still runs in its own thread. If the driver exits cleanly
then the AM will exit anyway since it joins on the driver thread, in which case
the exit code will still be 0. On the other hand, if the driver failed with an
exception, then we already handle that by finishing with non-zero exit code in
`startUserThread`. Listening on the disassociated event is just one way we
detect whether the driver has exited, and the behavior should be the same
across deploy modes.
By the way, I'm not saying that this patch is incorrect in its current
state. I just don't find the `isDriver` check in `AMActor` necessary because
the ultimate behavior should be the same without it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
