Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/715#discussion_r12515152
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -259,19 +238,30 @@ private[spark] class Executor(
}
case t: Throwable => {
- val serviceTime = System.currentTimeMillis() - taskStart
- val metrics = attemptedTask.flatMap(t => t.metrics)
- for (m <- metrics) {
- m.executorRunTime = serviceTime
- m.jvmGCTime = gcTime - startGCTime
- }
- val reason = ExceptionFailure(t.getClass.getName, t.toString,
t.getStackTrace, metrics)
- execBackend.statusUpdate(taskId, TaskState.FAILED,
ser.serialize(reason))
+ // Attempt to exit cleanly by informing the driver of our
failure.
+ // If anything goes wrong (or this was a fatal exception), we
will delegate to
+ // the default uncaught exception handler, which will terminate
the Executor.
+ try {
+ logError("Exception in task ID " + taskId, t)
+
+ val serviceTime = System.currentTimeMillis() - taskStart
+ val metrics = attemptedTask.flatMap(t => t.metrics)
+ for (m <- metrics) {
+ m.executorRunTime = serviceTime
+ m.jvmGCTime = gcTime - startGCTime
+ }
+ val reason = ExceptionFailure(t.getClass.getName, t.toString,
t.getStackTrace, metrics)
+ execBackend.statusUpdate(taskId, TaskState.FAILED,
ser.serialize(reason))
- // TODO: Should we exit the whole executor here? On the one
hand, the failed task may
- // have left some weird state around depending on when the
exception was thrown, but on
- // the other hand, maybe we could detect that when future tasks
fail and exit then.
- logError("Exception in task ID " + taskId, t)
+ // Don't forcibly exit unless the exception was inherently
fatal, to avoid
+ // stopping other tasks unnecessarily.
+ if (Utils.isFatalError(t)) {
+ ExecutorUncaughtExceptionHandler.uncaughtException(t)
+ }
+ } catch {
+ case t2: Throwable =>
+ ExecutorUncaughtExceptionHandler.uncaughtException(t2)
--- End diff --
Actually just realized we basically already have that comment, just
interpreted in a different way :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
