Github user szhem commented on a diff in the pull request:
https://github.com/apache/spark/pull/23083#discussion_r235757779
--- Diff: core/src/main/scala/org/apache/spark/TaskContextImpl.scala ---
@@ -99,6 +99,13 @@ private[spark] class TaskContextImpl(
this
}
+ override def remoteTaskCompletionListener(listener:
TaskCompletionListener)
+ : this.type = synchronized {
+ onCompleteCallbacks -= listener
--- End diff --
Replaced ArrayBuffer with LinkedHashSet. Thank you!
Another interesting question is why these collections are traversed in the
reverse order in
[invokeLisneters](https://github.com/apache/spark/pull/23083/files#diff-df60223d4208aff4c67335528a55154cR135)
like the following
```scala
private def invokeListeners[T](
listeners: Seq[T],
name: String,
error: Option[Throwable])(
callback: T => Unit): Unit = {
val errorMsgs = new ArrayBuffer[String](2)
// Process callbacks in the reverse order of registration
listeners.reverse.foreach { listener =>
...
}
}
```
I believe @hvanhovell could help to understand. @hvanhovell Could you
please remind why task completion and error listeners are traversed in the
reverse order (you seem to the the one who added the corresponding line)?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
