Github user LuciferYang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22149#discussion_r235257700
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformationExec.scala
---
@@ -308,6 +308,12 @@ private class ScriptTransformationWriterThread(
}
threwException = false
} catch {
+ // TaskKilledException should not be thrown again, otherwise it will
be captured by
+ // SparkUncaughtExceptionHandler, then Executor will exit because of
TaskKilledException.
+ case e: TaskKilledException =>
--- End diff --
Sorry, maybe I didn't express it clearly.
There is no essential difference between `ScriptTransformationExec` and
other operators in exception processing, exceptions handle by `catch` block of
`Executor.TaskRunner#run` method.
The key point is`ScriptTransformationExec` create a new thread named
`ScriptTransformationWriterThread `, it belongs to main threadgroup,
Exception throw from this thread cannot catch by `catch` block of
`Executor.TaskRunner#run` method, and it will handle by
`SparkUncaughtExceptionHandler` which we register in `Executor` at noLocal run
model, `SparkUncaughtExceptionHandler` will call
`System.exit(SparkExitCode.UNCAUGHT_EXCEPTION)` when `case _ if
exitOnUncaughtException`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
