Spark streaming

Hi, I'm pulling some twitter data and trying to save the data into persistent table.This is the code written. case class Tweet(createdAt:Long, text:String) twt.map(status=> Tweet(status.getCreatedAt().getTime()/1000, status.getText()) ).foreachRDD(rdd=>

spark streaming

Hi, Im new to Spark. I need to pull Twitter data based on some keyword using scala and store them in spark sql table. Can anyone help me out to work on this? Im using scala 1.6.0 Thanks and regrads, siva

Re: spark streaming

Have you looked at the examples ? examples/src/main/java/org/apache/spark/examples/streaming/JavaTwitterHashTagJoinSentiments.java examples/src/main/scala/org/apache/spark/examples/streaming/TwitterAlgebirdCMS.scala

Re: spark streaming

Siva: How did you build your project ? You can use mvn denpendency:tree to find out the version of spark-streaming-twitter or take a look at ./external/twitter/pom.xml ls ~/.m2/repository/org/apache/spark/spark-streaming-twitter_2.10/ 1.4.0-SNAPSHOT/ 1.5.0-SNAPSHOT/ 1.5.2

Re: spark streaming

Did you try using Twitter4J? http://twitter4j.org/en/index.html On Fri, Feb 5, 2016 at 5:20 AM, siva kumar wrote: > Hi , > Im trying to pull data from twitter and do some processing out of > it. I found this code > > import org.apache.spark.streaming._import >

spark streaming

Hi , Im trying to pull data from twitter and do some processing out of it. I found this code import org.apache.spark.streaming._import org.apache.spark.streaming.twitter._import org.apache.spark.storage.StorageLevelimport scala.io.Sourceimport scala.collection.mutable.HashMapimport