Re: Didn't receive video stream data into spark through kafka

2018-07-25 Thread Biswajit Ghosh
Hi, Thank you for your support! everything is working except can't figure out how to pass the video frame( serialize in a pickle file) to spark. My problem is during loading the pickle file stream got an EOF error into spark stream context. I'm suspecting that due to large file size TCP broke

Re: Didn't receive video stream data into spark through kafka

2018-07-25 Thread Attila Sasvári
Hi Biswajit, Can you please provide more information: - What other symptoms do you see? Are all your Kafka brokers up and running? - What replication did you set for offsets.topic.replication.factor (i.e. replication factor of __consumer_offsets) in your Kafka broker's config ? What is set for

Re: Didn't receive video stream data into spark through kafka

2018-07-24 Thread Biswajit Ghosh
Yes, I have double check that. On 24 July 2018 at 19:20, Aman Rastogi wrote: > Is your topic same in both the case? > > On Tue, 24 Jul 2018, 19:15 Biswajit Ghosh, > wrote: > > > Hi team, > > > > I got an issue while integrating with the spark streaming using pyspark, > I > > did receive the

Re: Didn't receive video stream data into spark through kafka

2018-07-24 Thread Aman Rastogi
Is your topic same in both the case? On Tue, 24 Jul 2018, 19:15 Biswajit Ghosh, wrote: > Hi team, > > I got an issue while integrating with the spark streaming using pyspark, I > did receive the video stream data in a different consumer subscribe to the > same topic. > > Works fine with this

Didn't receive video stream data into spark through kafka

2018-07-24 Thread Biswajit Ghosh
Hi team, I got an issue while integrating with the spark streaming using pyspark, I did receive the video stream data in a different consumer subscribe to the same topic. Works fine with this command : *./kafka-console-consumer.sh --bootstrap-server :9092 --topic spark-streaming-consumer