GitHub user concretevitamin opened a pull request:
https://github.com/apache/spark/pull/251
Blockingly drain all events in LiveListenerBus#stop().
This PR attempts to implement the proposal discussed in #221.
Some notes:
- Also, move bus.stop() call to the end of sc.stop(), so that slow
listeners would not affect the cleaning of other states.
- LiveListenerBus.scala: newline changes from CRLF to LF.
- Add a test to track the draining behavior. I've (1) used the patch to
test my application mentioned in the old PR, and it worked as expected, and
(2) run the new test without this patch, and it fails as expected.
@kayousterhout @pwendell @andrewor14: do you guys mind reviewing this?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/concretevitamin/spark listenerbus
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/251.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #251
----
commit a49dc74802738c318279e988bd699c8211d50670
Author: Zongheng Yang <[email protected]>
Date: 2014-03-27T05:36:45Z
Blockingly drain all events in LiveListenerBus#stop().
Also, move bus.stop() call to the end of sc.stop(), so that slow
listeners would not affect the cleaning of other states.
commit 36bcc4beccaa18691f76222c6cfe9a3526595a86
Author: Zongheng Yang <[email protected]>
Date: 2014-03-27T05:47:36Z
LiveListenerBus.scala: newline changes from CRLF to LF.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
