I tried all combinations of spark-cassandra connector. Didn't work. Finally, I downgraded spark to 1.5.1 and now it works. LCassa
On Wed, May 18, 2016 at 11:11 AM, Mohammed Guller <moham...@glassbeam.com> wrote: > As Ben mentioned, Spark 1.5.2 does work with C*. Make sure that you are > using the correct version of the Spark Cassandra Connector. > > > > > > Mohammed > > Author: Big Data Analytics with Spark > <http://www.amazon.com/Big-Data-Analytics-Spark-Practitioners/dp/1484209656/> > > > > *From:* Ben Slater [mailto:ben.sla...@instaclustr.com] > *Sent:* Tuesday, May 17, 2016 11:00 PM > *To:* user@cassandra.apache.org; Mohammed Guller > *Cc:* user > > *Subject:* Re: Accessing Cassandra data from Spark Shell > > > > It definitely should be possible for 1.5.2 (I have used it with > spark-shell and cassandra connector with 1.4.x). The main trick is in > lining up all the versions and building an appropriate connector jar. > > > > Cheers > > Ben > > > > On Wed, 18 May 2016 at 15:40 Cassa L <lcas...@gmail.com> wrote: > > Hi, > > I followed instructions to run SparkShell with Spark-1.6. It works fine. > However, I need to use spark-1.5.2 version. With it, it does not work. I > keep getting NoSuchMethod Errors. Is there any issue running Spark Shell > for Cassandra using older version of Spark? > > > > > > Regards, > > LCassa > > > > On Tue, May 10, 2016 at 6:48 PM, Mohammed Guller <moham...@glassbeam.com> > wrote: > > Yes, it is very simple to access Cassandra data using Spark shell. > > > > Step 1: Launch the spark-shell with the spark-cassandra-connector package > > $SPARK_HOME/bin/spark-shell --packages > com.datastax.spark:spark-cassandra-connector_2.10:1.5.0 > > > > Step 2: Create a DataFrame pointing to your Cassandra table > > val dfCassTable = sqlContext.read > > > .format("org.apache.spark.sql.cassandra") > > .options(Map( > "table" -> "your_column_family", "keyspace" -> "your_keyspace")) > > .load() > > > > From this point onward, you have complete access to the DataFrame API. You > can even register it as a temporary table, if you would prefer to use > SQL/HiveQL. > > > > Mohammed > > Author: Big Data Analytics with Spark > <http://www.amazon.com/Big-Data-Analytics-Spark-Practitioners/dp/1484209656/> > > > > *From:* Ben Slater [mailto:ben.sla...@instaclustr.com] > *Sent:* Monday, May 9, 2016 9:28 PM > *To:* user@cassandra.apache.org; user > *Subject:* Re: Accessing Cassandra data from Spark Shell > > > > You can use SparkShell to access Cassandra via the Spark Cassandra > connector. The getting started article on our support page will probably > give you a good steer to get started even if you’re not using Instaclustr: > https://support.instaclustr.com/hc/en-us/articles/213097877-Getting-Started-with-Instaclustr-Spark-Cassandra- > > > > Cheers > > Ben > > > > On Tue, 10 May 2016 at 14:08 Cassa L <lcas...@gmail.com> wrote: > > Hi, > > Has anyone tried accessing Cassandra data using SparkShell? How do you do > it? Can you use HiveContext for Cassandra data? I'm using community version > of Cassandra-3.0 > > > > Thanks, > > LCassa > > -- > > ———————— > > Ben Slater > > Chief Product Officer, Instaclustr > > +61 437 929 798 > > > > -- > > ———————— > > Ben Slater > > Chief Product Officer, Instaclustr > > +61 437 929 798 >