Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/221#issuecomment-38608812
I think SparkListenerEvents should only be posted from within Spark, and
exposing the LiveListenerBus to the user violates this. I can think of two
workarounds for this:
1) As Kay suggested, optionally drain all events in LiveListenerBus.stop()
until a configurable timeout has exceeded.
2) We could expose some sort of boolean flag instead of the entire listener
bus, perhaps something like `sc.readyToStop`, which returns true only if there
are no more events left in the LiveListenerBus.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
