Github user Sephiroth-Lin commented on the pull request:
https://github.com/apache/spark/pull/4412#issuecomment-73190453
@srowen we run a process as a service which will not stop. In this service
process we will create SparkContext and run job and then stop it, because we
only call sc.stop but not exit this service process so the tmp dirs created by
HttpFileServer and SparkEnv will not be deleted after SparkContext is stopped,
and this will lead to creating too many tmp dirs if we create many SparkContext
to run job in this service process.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
