Jacek Laskowski created SPARK-12339:
---------------------------------------
Summary: NullPointerException on stage kill from web UI
Key: SPARK-12339
URL: https://issues.apache.org/jira/browse/SPARK-12339
Project: Spark
Issue Type: Bug
Components: Web UI
Affects Versions: 1.6.0
Reporter: Jacek Laskowski
The following message is in the logs after killing a stage:
{code}
scala> INFO Executor: Executor killed task 1.0 in stage 7.0 (TID 33)
INFO Executor: Executor killed task 0.0 in stage 7.0 (TID 32)
WARN TaskSetManager: Lost task 1.0 in stage 7.0 (TID 33, localhost): TaskKilled
(killed intentionally)
WARN TaskSetManager: Lost task 0.0 in stage 7.0 (TID 32, localhost): TaskKilled
(killed intentionally)
INFO TaskSchedulerImpl: Removed TaskSet 7.0, whose tasks have all completed,
from pool
ERROR LiveListenerBus: Listener SQLListener threw an exception
java.lang.NullPointerException
at
org.apache.spark.sql.execution.ui.SQLListener.onTaskEnd(SQLListener.scala:167)
at
org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:42)
at
org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
at
org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
at
org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:55)
at
org.apache.spark.util.AsynchronousListenerBus.postToAll(AsynchronousListenerBus.scala:37)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(AsynchronousListenerBus.scala:80)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(AsynchronousListenerBus.scala:65)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(AsynchronousListenerBus.scala:65)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(AsynchronousListenerBus.scala:64)
at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1169)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1.run(AsynchronousListenerBus.scala:63)
ERROR LiveListenerBus: Listener SQLListener threw an exception
java.lang.NullPointerException
at
org.apache.spark.sql.execution.ui.SQLListener.onTaskEnd(SQLListener.scala:167)
at
org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:42)
at
org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
at
org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
at
org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:55)
at
org.apache.spark.util.AsynchronousListenerBus.postToAll(AsynchronousListenerBus.scala:37)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(AsynchronousListenerBus.scala:80)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(AsynchronousListenerBus.scala:65)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(AsynchronousListenerBus.scala:65)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(AsynchronousListenerBus.scala:64)
at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1169)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1.run(AsynchronousListenerBus.scala:63)
{code}
To reproduce, start a job and kill the stage from web UI, e.g.:
{code}
val rdd = sc.parallelize(0 to 9, 2)
rdd.mapPartitionsWithIndex { case (n, it) => Thread.sleep(10 * 1000); it }.count
{code}
Go to web UI and in Stages tab click "kill" for the stage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
