Re: Problem getting pyspark-cassandra and pyspark working

2015-02-16 Thread Mohamed Lrhazi
Will do. Thanks a lot. On Mon, Feb 16, 2015 at 7:20 PM, Davies Liu wrote: > Can you try the example in pyspark-cassandra? > > If not, you could create a issue there. > > On Mon, Feb 16, 2015 at 4:07 PM, Mohamed Lrhazi > wrote: > > So I tired building the connector from: > > https://github.com/

Re: Problem getting pyspark-cassandra and pyspark working

2015-02-16 Thread Davies Liu
Can you try the example in pyspark-cassandra? If not, you could create a issue there. On Mon, Feb 16, 2015 at 4:07 PM, Mohamed Lrhazi wrote: > So I tired building the connector from: > https://github.com/datastax/spark-cassandra-connector > > which seems to include the java class referenced in t

Re: Problem getting pyspark-cassandra and pyspark working

2015-02-16 Thread Mohamed Lrhazi
So I tired building the connector from: https://github.com/datastax/spark-cassandra-connector which seems to include the java class referenced in the error message: [root@devzero spark]# unzip -l spark-cassandra-connector/spark-cassandra-connector-java/target/scala-2.10/spark-cassandra-connector-

Re: Problem getting pyspark-cassandra and pyspark working

2015-02-16 Thread Mohamed Lrhazi
Oh, I don't know. thanks a lot Davies, gonna figure that out now On Mon, Feb 16, 2015 at 5:31 PM, Davies Liu wrote: > It also need the Cassandra jar: > com.datastax.spark.connector.CassandraJavaUtil > > Is it included in /spark/pyspark-cassandra-0.1-SNAPSHOT.jar ? > > > > On Mon, Feb 16, 20

Re: Problem getting pyspark-cassandra and pyspark working

2015-02-16 Thread Davies Liu
It also need the Cassandra jar: com.datastax.spark.connector.CassandraJavaUtil Is it included in /spark/pyspark-cassandra-0.1-SNAPSHOT.jar ? On Mon, Feb 16, 2015 at 1:20 PM, Mohamed Lrhazi wrote: > Yes, am sure the system cant find the jar.. but how do I fix that... my > submit command includ

Re: Problem getting pyspark-cassandra and pyspark working

2015-02-16 Thread Mohamed Lrhazi
Yes, am sure the system cant find the jar.. but how do I fix that... my submit command includes the jar: /spark/bin/spark-submit --py-files /spark/pyspark_cassandra.py --jars /spark/pyspark-cassandra-0.1-SNAPSHOT.jar --driver-class-path /spark/pyspark-cassandra-0.1-SNAPSHOT.jar /spark/test2.py an

Re: Problem getting pyspark-cassandra and pyspark working

2015-02-16 Thread Davies Liu
It seems that the jar for cassandra is not loaded, you should have them in the classpath. On Mon, Feb 16, 2015 at 12:08 PM, Mohamed Lrhazi wrote: > Hello all, > > Trying the example code from this package > (https://github.com/Parsely/pyspark-cassandra) , I always get this error... > > Can you se

Problem getting pyspark-cassandra and pyspark working

2015-02-16 Thread Mohamed Lrhazi
Hello all, Trying the example code from this package ( https://github.com/Parsely/pyspark-cassandra) , I always get this error... Can you see what I am doing wrong? from googling arounf it seems to be that the jar is not found somehow... The spark log shows the JAR was processed at least. Thank