Github user concretevitamin commented on the pull request:
https://github.com/apache/spark/pull/221#issuecomment-38652864
Thanks for the proposals everyone. For our case a Spark-enforced timeout
might not be ideal, and only moving `listenerBus.stop()` to the end of
`sc.stop()` probably does not guarantee draining. So how about exposing a
binary flag from `SparkContext`, plus moving `listenerBus.stop()`? To recap,
this makes sure other states / info get cleaned up in `sc.stop()` and do not
get stuck because of the listener bus, and the user application can have the
flexibility to poll the binary drained flag.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
