Github user liorregev commented on the issue:
https://github.com/apache/spark/pull/17986
Actually I ran into a problem with this not getting cleaned up.
After your explanation I can understand why it wasn't deleted.
I am running spark on EMR and the easiest way to programmatically submit
applications to the cluster was to create an HTTP service that accepts the
application details and programmatically calls SparkSubmit.main so the process
never really exits.
I managed to solve this with spark.yarn.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
