Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/1938#discussion_r16266374
--- Diff: core/src/main/scala/org/apache/spark/TaskContext.scala ---
@@ -39,13 +47,45 @@ class TaskContext(
def splitId = partitionId
// List of callback functions to execute when the task completes.
- @transient private val onCompleteCallbacks = new ArrayBuffer[() => Unit]
+ @transient private val onCompleteCallbacks = new
ArrayBuffer[TaskCompletionListener]
// Whether the corresponding task has been killed.
- @volatile var interrupted: Boolean = false
+ @volatile private var interrupted: Boolean = false
+
+ // Whether the task has completed.
+ @volatile private var completed: Boolean = false
+
+ /** Checks whether the task has completed. */
+ def isCompleted: Boolean = completed
- // Whether the task has completed, before the onCompleteCallbacks are
executed.
- @volatile var completed: Boolean = false
+ /** Checks whether the task has been killed. */
+ def isInterrupted: Boolean = interrupted
+
+ // TODO: Also track whether the task has completed successfully or with
exception.
+
+ /**
+ * Add a (Java friendly) listener to be executed on task completion.
+ * This will be called in all situation - success, failure, or
cancellation.
+ *
+ * An example use is for HadoopRDD to register a callback to close the
input stream.
+ */
+ def addTaskCompletionListener(listener: TaskCompletionListener):
this.type = {
+ onCompleteCallbacks += listener
+ this
+ }
+
+ /**
+ * Add a listener in the form of a Scala closure to be executed on task
completion.
+ * This will be called in all situation - success, failure, or
cancellation.
+ *
+ * An example use is for HadoopRDD to register a callback to close the
input stream.
+ */
+ def addTaskCompletionListener(f: TaskContext => Unit): this.type = {
--- End diff --
yea, implicits are evil.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
