Github user LuciferYang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22149#discussion_r235246544
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformationExec.scala
---
@@ -308,6 +308,12 @@ private class ScriptTransformationWriterThread(
}
threwException = false
} catch {
+ // TaskKilledException should not be thrown again, otherwise it will
be captured by
+ // SparkUncaughtExceptionHandler, then Executor will exit because of
TaskKilledException.
+ case e: TaskKilledException =>
--- End diff --
Thanks for your reply @cloud-fan , the error log and problem analysis of
one bad case I paste in https://issues.apache.org/jira/browse/SPARK-25158, in
that case `.UnsafeInMemorySorter$SortedIterator.loadNext` throw
`TaskKilledException`, and in another case `TaskKilledException` throw by
`InterruptibleIterator.hasNext` , and from the error stack line 281
`iter.map(outputProjection).foreach ` is the stack root.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
