[
https://issues.apache.org/jira/browse/SPARK-12945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15514850#comment-15514850
]
Josh Rosen commented on SPARK-12945:
------------------------------------
In general, the {{JobProgressListener}} should not be throwing exceptions. Even
if those exceptions don't cause job failures or errors it's possible that they
may indicate the presence of a bug which could impact the consistency of the
metrics displayed on the Spark UI.
It would be helpful to know if there's an easy way to reliably reproduce this
exception so that we can fix it.
> ERROR LiveListenerBus: Listener JobProgressListener threw an exception
> ----------------------------------------------------------------------
>
> Key: SPARK-12945
> URL: https://issues.apache.org/jira/browse/SPARK-12945
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.6.0
> Environment: Linux, yarn-client
> Reporter: Tristan
> Priority: Minor
>
> Seeing this a lot; not sure if it is a problem or spurious error (I recall
> this was an ignorable issue in previous version). The UI seems to be working
> fine:
> ERROR LiveListenerBus: Listener JobProgressListener threw an exception
> java.lang.NullPointerException
> at
> org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onTaskEnd$1.apply(JobProgressListener.scala:361)
> at
> org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onTaskEnd$1.apply(JobProgressListener.scala:360)
> at scala.collection.immutable.List.foreach(List.scala:318)
> at
> scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
> at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
> at
> org.apache.spark.ui.jobs.JobProgressListener.onTaskEnd(JobProgressListener.scala:360)
> at
> org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:42)
> at
> org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
> at
> org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
> at
> org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:55)
> at
> org.apache.spark.util.AsynchronousListenerBus.postToAll(AsynchronousListenerBus.scala:37)
> at
> org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(AsynchronousListenerBus.scala:80)
> at
> org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(AsynchronousListenerBus.scala:65)
> at
> org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(AsynchronousListenerBus.scala:65)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
> at
> org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(AsynchronousListenerBus.scala:64)
> at
> org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1180)
> at
> org.apache.spark.util.AsynchronousListenerBus$$anon$1.run(AsynchronousListenerBus.scala:63)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
