Github user piaozhexiu commented on the pull request:
https://github.com/apache/spark/pull/5343#issuecomment-90193027
Hi @tgravescs , thank you for your comment.
Changed the title as you suggested.
> Relying on the fact the client goes away kills the job when its running
on the cluster seems unreliable to me.
Since I am only killing AM inside the shutdown hook, this shouldn't be the
case, no? When network connection between client and AM is lost, it won't kill
AM as long as client continues to run.
> But it seems like it would be better to ask for status of the app and
kill it via yarn.
I also added a check for the status of AM prior to killing it. Will that
address your concern?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
