Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/16743
There's no way to set the app state currently because in client mode (which
includes local) Spark lacks an API to do that. So the code guesses that since
"stop" was called, the app finished successfully. In cluster mode, Spark has
more info and can propagate the proper status.
It might be possible to detect failure in client mode by checking the exit
code of the underlying spark-submit process. But the current code here is
wrong. A task failing does not mean the application failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
