I've been running into the following error on creating the streaming context
using checkpoint files.
This error occurs quite often when I stop and re-start the job.
Any suggestions?
14-06-09 23:47:59 WARN CheckpointReader:91 - Error reading checkpoint from
file
file:/Users/kanwaldeep.dang/git/cidhbase/sinet-ingestion/sinet-spark-streaming/streamAggregateSparkCheckpoint/checkpoint-1402382820000
*java.lang.NullPointerException
at
org.apache.spark.broadcast.HttpBroadcast.readObject(HttpBroadcast.scala:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:979)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1873)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
* at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1685)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at
java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:479)
at
org.apache.spark.streaming.DStreamGraph.readObject(DStreamGraph.scala:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:979)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1873)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1970)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1895)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)
at
org.apache.spark.streaming.CheckpointReader$$anonfun$read$2.apply(Checkpoint.scala:252)
at
org.apache.spark.streaming.CheckpointReader$$anonfun$read$2.apply(Checkpoint.scala:240)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34)
at
org.apache.spark.streaming.CheckpointReader$.read(Checkpoint.scala:240)
at
org.apache.spark.streaming.StreamingContext$.getOrCreate(StreamingContext.scala:530)
at
com.snc.sinet.streaming.StreamAggregates$.start(StreamAggregates.scala:58)
at
com.snc.sinet.streaming.StreamAggregates$.main(StreamAggregates.scala:46)
at com.snc.sinet.streaming.StreamAggregates.main(StreamAggregates.scala)
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/NullPointerException-on-reading-checkpoint-files-tp7306.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
