Github user ted-yu commented on the pull request:
https://github.com/apache/spark/pull/9546#issuecomment-156867526
Cloning git repo was extremely slow.
Here is proposed fix:
```
diff --git
a/core/src/main/scala/org/apache/spark/util/AsynchronousListenerBus.scala
b/core/src/main/scala/org/apache/spark/util/AsynchronousListenerBus.scala
index b3b54af..cc58bc5 100644
---
a/core/src/main/scala/org/apache/spark/util/AsynchronousListenerBus.scala
+++
b/core/src/main/scala/org/apache/spark/util/AsynchronousListenerBus.scala
@@ -56,19 +56,24 @@ private[spark] abstract class AsynchronousListenerBus[L
<: AnyRef, E](name: Stri
// A counter that represents the number of events produced and consumed
in the queue
private val eventLock = new Semaphore(0)
+ // limit on the number of events to process before exiting. -1 means no
limit
+ private val eventLimit = -1
private val listenerThread = new Thread(name) {
setDaemon(true)
override def run(): Unit = Utils.tryOrStopSparkContext(sparkContext) {
- while (true) {
+ while (eventLimit != 0) {
eventLock.acquire()
self.synchronized {
processingEvent = true
}
try {
if (stopped.get()) {
- // Get out of the while loop and shutdown the daemon thread
- return
+ eventLimit = eventQueue.size
+ if (eventLimit == 0) {
+ // Get out of the while loop and shutdown the daemon thread
+ return
+ }
}
val event = eventQueue.poll
assert(event != null, "event queue was empty but the listener
bus was not stopped")
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
