[
https://issues.apache.org/jira/browse/SPARK-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955046#comment-13955046
]
wangfei commented on SPARK-1361:
--------------------------------
this is an order issue of construction and starting of the taskScheduler and
the dagScheduler in SparkContext. now taskScheduler start before
dagScheduler, so executor backend may register executor even before
dagScheduler(or the eventProcessActor) has not initialized. in this situation
it will throw NPE
> DAGScheduler throws NullPointerException occasionally
> -----------------------------------------------------
>
> Key: SPARK-1361
> URL: https://issues.apache.org/jira/browse/SPARK-1361
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 0.9.0
> Reporter: wangfei
> Fix For: 1.0.0
>
>
> DAGScheduler throws this NullPointerException below occasionally when running
> spark apps.
> java.lang.NullPointerException
> at
> org.apache.spark.scheduler.DAGScheduler.executorAdded(DAGScheduler.scala:186)
> at
> org.apache.spark.scheduler.TaskSchedulerImpl.executorAdded(TaskSchedulerImpl.scala:409)
> at
> org.apache.spark.scheduler.TaskSchedulerImpl$$anonfun$resourceOffers$1.apply(TaskSchedulerImpl.scala:210)
> at
> org.apache.spark.scheduler.TaskSchedulerImpl$$anonfun$resourceOffers$1.apply(TaskSchedulerImpl.scala:206)
> at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:73)
> at
> org.apache.spark.scheduler.TaskSchedulerImpl.resourceOffers(TaskSchedulerImpl.scala:206)
> at
> org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend$DriverActor.makeOffers(CoarseGrainedSchedulerBackend.scala:130)
> at
> org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend$DriverActor$$anonfun$receive$1.applyOrEls
--
This message was sent by Atlassian JIRA
(v6.2#6252)
