Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/2311#discussion_r17946154
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -91,7 +94,11 @@ private[spark] class ApplicationMaster(args:
ApplicationMasterArguments,
if (sc != null) {
logInfo("Invoking sc stop from shutdown hook")
sc.stop()
- finish(FinalApplicationStatus.SUCCEEDED)
+ }
+
+ // Shuts down the AM.
+ if (!finished) {
--- End diff --
Basically there are a few cases:
- If the machine with the AM dies, then sure, YARN could relaunch it
(hopefully that is user-configurable in case their code has weird side effects)
- If the driver returns 0, it's finished
- If the driver crashes for another reason, it could be a bug in the
driver; I'm not sure we should just resubmit in that case
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
