thanks, Mark.
On Thu, Oct 17, 2013 at 6:36 PM, Mark Hamstra <[email protected]>wrote: > SNAPSHOTs are not fixed versions, but are floating names associated with > whatever is the most recent code. So, Spark 0.8.0 is the current released > version of Spark, which is exactly the same today as it was yesterday, and > will be the same thing forever. Spark 0.8.1-SNAPSHOT is whatever is > currently in branch-0.8. It changes every time new code is committed to > that branch (which should be just bug fixes and the few additional features > that we wanted to get into 0.8.0, but that didn't quite make it.) Not too > long from now there will be a release of Spark 0.8.1, at which time the > SNAPSHOT will got to 0.8.2 and 0.8.1 will be forever frozen. Meanwhile, > the wild new development is taking place on the master branch, and whatever > is currently in that branch becomes 0.9.0-SNAPSHOT. This could be quite > different from day to day, and there are no guarantees that things won't be > broken in 0.9.0-SNAPSHOT. Several months from now there will be a release > of Spark 0.9.0 (unless the decision is made to bump the version to 1.0.0), > at which point the SNAPSHOT goes to 0.9.1 and the whole process advances to > the next phase of development. > > The short answer is that releases are stable, SNAPSHOTs are not, and > SNAPSHOTs that aren't on maintenance branches can break things. You make > your choice of which to use and pay the consequences. > > > On Thu, Oct 17, 2013 at 3:18 PM, dachuan <[email protected]> wrote: > >> yeah, I mean 0.9.0-SNAPSHOT. I use git clone and that's what I got.. >> what's the difference? I mean SNAPSHOT and non-SNAPSHOT. >> >> >> On Thu, Oct 17, 2013 at 6:15 PM, Mark Hamstra <[email protected]>wrote: >> >>> Of course, you mean 0.9.0-SNAPSHOT. There is no Spark 0.9.0, and won't >>> be for several months. >>> >>> >>> >>> On Thu, Oct 17, 2013 at 3:11 PM, dachuan <[email protected]> wrote: >>> >>>> I'm sorry if this doesn't answer your question directly, but I have >>>> tried spark 0.9.0 and hdfs 1.0.4 just now, it works.. >>>> >>>> >>>> On Thu, Oct 17, 2013 at 6:05 PM, Koert Kuipers <[email protected]>wrote: >>>> >>>>> after upgrading from spark 0.7 to spark 0.8 i can no longer access any >>>>> files on HDFS. >>>>> i see the error below. any ideas? >>>>> >>>>> i am running spark standalone on a cluster that also has CDH4.3.0 and >>>>> rebuild spark accordingly. the jars in lib_managed look good to me. >>>>> >>>>> i noticed similar errors in the mailing list but found no suggested >>>>> solutions. >>>>> >>>>> thanks! koert >>>>> >>>>> >>>>> 13/10/17 17:43:23 ERROR Executor: Exception in task ID 0 >>>>> java.io.EOFException >>>>> at >>>>> java.io.ObjectInputStream$BlockDataInputStream.readFully(ObjectInputStream.java:2703) >>>>> at java.io.ObjectInputStream.readFully(ObjectInputStream.java:1008) >>>>> at >>>>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:68) >>>>> at >>>>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:106) >>>>> at org.apache.hadoop.io.UTF8.readChars(UTF8.java:258) >>>>> at org.apache.hadoop.io.UTF8.readString(UTF8.java:250) >>>>> at org.apache.hadoop.mapred.FileSplit.readFields(FileSplit.java:87) >>>>> at >>>>> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:280) >>>>> at >>>>> org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75) >>>>> at >>>>> org.apache.spark.SerializableWritable.readObject(SerializableWritable.scala:39) >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> at >>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>>> at >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>> at >>>>> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969) >>>>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1852) >>>>> at >>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1756) >>>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1326) >>>>> at >>>>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1950) >>>>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1874) >>>>> at >>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1756) >>>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1326) >>>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) >>>>> at >>>>> org.apache.spark.scheduler.ResultTask.readExternal(ResultTask.scala:135) >>>>> at >>>>> java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1795) >>>>> at >>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1754) >>>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1326) >>>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) >>>>> at >>>>> org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:39) >>>>> at >>>>> org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:61) >>>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:153) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>>>> at java.lang.Thread.run(Thread.java:662) >>>>> >>>>> >>>> >>>> >>>> -- >>>> Dachuan Huang >>>> Cellphone: 614-390-7234 >>>> 2015 Neil Avenue >>>> Ohio State University >>>> Columbus, Ohio >>>> U.S.A. >>>> 43210 >>>> >>> >>> >> >> >> -- >> Dachuan Huang >> Cellphone: 614-390-7234 >> 2015 Neil Avenue >> Ohio State University >> Columbus, Ohio >> U.S.A. >> 43210 >> > > -- Dachuan Huang Cellphone: 614-390-7234 2015 Neil Avenue Ohio State University Columbus, Ohio U.S.A. 43210
