fsamuel-bs commented on pull request #29977: URL: https://github.com/apache/spark/pull/29977#issuecomment-705691969
> You also have other situations where tasks are killed, which luckily I guess the TaskContext does have a isInterrupted field for that. The executor ends up making some pass/fail decisions that go to the scheduler I see. For my use-case, I don't really care about the distinction of `success` vs `fail`, only that a method will be called at least once when the task finishes in the same thread the task is executed on (for me to dismantle the tracing stuff I've built up `onTaskStart`). Could expose instead an `onTaskCompleted` method, which does not leak the fact that a task succeeded or failed. Or could move triggering `onTaskSucceeded`/`onTaskFailed` from the Executor. Think the latter is better, because it seems useful to have the distinction if a task succeeded or failed, but happy to implement whatever maintainers prefer. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
