Re: Saving to Cassandra from Spark Streaming

[Akhil Das]

You need to set the following jar (cassandra-connector
)
in the classpath like:

ssc.sparkContext.addJar("/path/to/spark-cassandra-connector_2.10-1.1.0-alpha3.jar")

Thanks
Best Regards

On Tue, Oct 28, 2014 at 10:09 PM, Gerard Maas  wrote:

> Looks like you're having some classpath issues.
> Are you providing your spark-cassandra-driver classes to your job?
>  sparkConf.setJars(Seq(jars...)) ?
>
> On Tue, Oct 28, 2014 at 5:34 PM, Harold Nguyen  wrote:
>
>> Hi all,
>>
>> I'm having trouble troubleshooting this particular block of code for
>> Spark Streaming and saving to Cassandra:
>>
>> val lines = ssc.socketTextStream(args(0), args(1).toInt,
>> StorageLevel.MEMORY_AND_DISK_SER)
>> val words = lines.flatMap(_.split(" "))
>> val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _)
>>
>> //-- Writing it to Cassandra
>> wordCounts.saveToCassandra("test", "kv", SomeColumns("key", "value"),
>> 1)
>>
>> Could you tell me where I'm going wrong ? Can I not call
>> wordCounts.saveToCassandra ?
>>
>> Here's the error:
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> com/datastax/spark/connector/mapper/ColumnMapper
>>
>> Thanks,
>>
>> Harold
>>
>
>

Re: Saving to Cassandra from Spark Streaming

[Gerard Maas]

Looks like you're having some classpath issues.
Are you providing your spark-cassandra-driver classes to your job?
 sparkConf.setJars(Seq(jars...)) ?

On Tue, Oct 28, 2014 at 5:34 PM, Harold Nguyen  wrote:

> Hi all,
>
> I'm having trouble troubleshooting this particular block of code for Spark
> Streaming and saving to Cassandra:
>
> val lines = ssc.socketTextStream(args(0), args(1).toInt,
> StorageLevel.MEMORY_AND_DISK_SER)
> val words = lines.flatMap(_.split(" "))
> val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _)
>
> //-- Writing it to Cassandra
> wordCounts.saveToCassandra("test", "kv", SomeColumns("key", "value"),
> 1)
>
> Could you tell me where I'm going wrong ? Can I not call
> wordCounts.saveToCassandra ?
>
> Here's the error:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> com/datastax/spark/connector/mapper/ColumnMapper
>
> Thanks,
>
> Harold
>

Saving to Cassandra from Spark Streaming

[Harold Nguyen]

Hi all,

I'm having trouble troubleshooting this particular block of code for Spark
Streaming and saving to Cassandra:

val lines = ssc.socketTextStream(args(0), args(1).toInt,
StorageLevel.MEMORY_AND_DISK_SER)
val words = lines.flatMap(_.split(" "))
val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _)

//-- Writing it to Cassandra
wordCounts.saveToCassandra("test", "kv", SomeColumns("key", "value"), 1)

Could you tell me where I'm going wrong ? Can I not call
wordCounts.saveToCassandra ?

Here's the error:

Exception in thread "main" java.lang.NoClassDefFoundError:
com/datastax/spark/connector/mapper/ColumnMapper

Thanks,

Harold