Re: Re: mismatched hdfs protocol

ok, i see i imported wrong jar files which only work well on default hadoop version 2014-06-05 bluejoe2008 From: prabeesh k Date: 2014-06-05 16:14 To: user Subject: Re: Re: mismatched hdfs protocol If you are not setting the Spark hadoop version, Spark built using default hadoop version

Re: Re: mismatched hdfs protocol

thank you! i am developping a java project in Eclipse IDE on Windows in which spark 1.0.0 libraries are imported and now i want to open HDFS files as input the hadoop version of HDFS is 2.4.0 2014-06-05 bluejoe2008 From: prabeesh k Date: 2014-06-05 13:23 To: user Subject: Re: mismatched hdfs

mismatched hdfs protocol

hi, all when my spark program accessed hdfs files an error happened: Exception in thread "main" org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4 it seems the client was trying to connect hadoop2 via an old hadoop protocol so my question is: h

Re: Re: is there any easier way to define a custom RDD in Java

i want to use spark to handle data from non-sql databases (RDF triple store, for example) however, i am not familiar with Scala so i want to know how to create a RdfTriplesRDD rapidly 2014-06-05 bluejoe2008 From: Patrick Wendell Date: 2014-06-05 01:25 To: user Subject: Re: is there any

is there any easier way to define a custom RDD in Java

hi, folks, is there any easier way to define a custom RDD in Java? I am wondering if I have to define a new java class which extends RDD from scratch? It is really a hard job for developers! 2014-06-04 bluejoe2008

Re: Re: IllegalArgumentException on calling KMeans.train()

thank you! 孟祥瑞 with your help i solved the problem. I constructed SparseVectors in a wrong way the first parameter of the constructor SparseVector(int size, int[] indices, double[] values) I mistaked it for the size of values 2014-06-04 bluejoe2008 From: Xiangrui Meng Date: 2014-06-04

IllegalArgumentException on calling KMeans.train()

generateDocVector() are invoked for 3 times... 2014-06-04 bluejoe2008

KMeans.train() throws NotSerializableException

a:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) my codes look like: JavaRDD docVectors = ...; int numClusters = 20; int numIterations = 20; KMeansModel clusters = KMeans.train(docVectors.rdd(), numClusters, numIterations); version: 1.0.0 2014-06-04 bluejoe2008

Re: Re: how to construct a ClassTag object as a method parameter in Java

spark 0.9.1 textInput is a JavaRDD object i am programming in Java 2014-06-03 bluejoe2008 From: Michael Armbrust Date: 2014-06-03 10:09 To: user Subject: Re: how to construct a ClassTag object as a method parameter in Java What version of Spark are you using? Also are you sure the type of

how to construct a ClassTag object as a method parameter in Java

hen how can i define such a value in Java? i really have no idea... best regards, bluejoe2008