Re: Spark Structured Streaming for Twitter Streaming data

Got it Thanks for the clarification TD ! On Thu, 1 Feb 2018 at 11:36 AM, Tathagata Das wrote: > The code uses the format "socket" which is only for text sent over a > simple socket, which is completely different from how Twitter APIs works. > So this wont work at

Re: Spark Structured Streaming for Twitter Streaming data

The code uses the format "socket" which is only for text sent over a simple socket, which is completely different from how Twitter APIs works. So this wont work at all. Fundamentally, for Structured Streaming, we have focused only on those streaming sources that have the capabilities record-level

Re: Spark Structured Streaming for Twitter Streaming data

Hi , I see ,Does that means Spark structured streaming doesn't work with Twitter streams ? I could see people used kafka or other streaming tools and used spark to process the data in structured streaming . The below doesn't work directly with Twitter Stream until I set up Kafka ? > import

Re: Spark Structured Streaming for Twitter Streaming data

Hello Divya, To add further clarification, the Apache Bahir does not have any Structured Streaming support for Twitter. It only has support for Twitter + DStreams. TD On Wed, Jan 31, 2018 at 2:44 AM, vermanurag wrote: > Twitter functionality is not part of Core

Re: Spark Structured Streaming for Twitter Streaming data

Twitter functionality is not part of Core Spark. We have successfully used the following packages from maven central in past org.apache.bahir:spark-streaming-twitter_2.11:2.2.0 Earlier there used to be a twitter package under spark, but I find that it has not been updated beyond Spark 1.6

Spark Structured Streaming for Twitter Streaming data

Hi, I am exploring the spark structured streaming . When turned to internet to understand about it I could find its more integrated with Kafka or other streaming tool like Kenesis. I couldnt find where we can use Spark Streaming API for twitter streaming data . Would be grateful ,if any body used