I wonder of it has to do with the number of contexts problem since it always completes with a partial build.
I’d like to get that in with 1.2.1 this weekend so maybe it will go away. On Feb 20, 2015, at 10:06 AM, Dmitriy Lyubimov <[email protected]> wrote: We had various operational configuration problems with snappy as well so had to disable it for now completely until somebody has time to figure it out (which has been like forever) On Thu, Feb 19, 2015 at 4:26 PM, Pat Ferrel <[email protected]> wrote: > It seems like after a clean install I fail some test in spark like the one > below. Then when I build again starting with -rf spark everything passes. > I’m not sure its always after a clean but happens often. > > Anyone running into this? > > - C = A cbind B, cogroup *** FAILED *** > org.apache.spark.SparkException: Job aborted due to stage failure: Task > 0 in stage 205.0 failed 1 times, most recent failure: Lost task 0.0 in > stage 205.0 (TID 442, localhost): java.io.IOException: PARSING_ERROR(2) > org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:78) > org.xerial.snappy.SnappyNative.uncompressedLength(Native Method) > org.xerial.snappy.Snappy.uncompressedLength(Snappy.java:545) > > org.xerial.snappy.SnappyInputStream.readFully(SnappyInputStream.java:125) > > org.xerial.snappy.SnappyInputStream.readHeader(SnappyInputStream.java:88) > > org.xerial.snappy.SnappyInputStream.<init>(SnappyInputStream.java:58) > > org.apache.spark.io.SnappyCompressionCodec.compressedInputStream(CompressionCodec.scala:128) > > org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:232) > > org.apache.spark.broadcast.TorrentBroadcast.readObject(TorrentBroadcast.scala:169) > sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > java.lang.reflect.Method.invoke(Method.java:606) > > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893) > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) > > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990) > > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915) > > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798) > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) > java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) > > org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:62) > > org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:87) > > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:159) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > java.lang.Thread.run(Thread.java:745) > Driver stacktrace: > at org.apache.spark.scheduler.DAGScheduler.org > $apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1185) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1174) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1173) > at > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) > at > org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1173) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688) > at scala.Option.foreach(Option.scala:236) > at > org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:688) > ... > - C = A cbind B, zip > >
