Re: What is the real difference between Kafka streaming and Spark Streaming?

m: "kant kodali" Date: 2017/6/12 06:41:27 To: "Mohammed Guller"; Cc: "user";"yohann jardin";"vaquar khan";"vincent gromakowski"; Subject: Re: What is the real difference between Kafka streaming and Spark Streaming? Also another d

Re: What is the real difference between Kafka streaming and Spark Streaming?

Continuous processing is still a work in progress. I would really like to at least have a basic version in Spark 2.3. The announcement about 2.2 is that we are planning to remove the experimental tag from Structured Streaming. On Thu, Jun 15, 2017 at 11:53 AM, kant kodali wrote: > vow! you cau

Re: What is the real difference between Kafka streaming and Spark Streaming?

vow! you caught the 007! Is continuous processing mode available in 2.2? The ticket says the target version is 2.3 but the talk in the Video says 2.2 and beyond so I am just curious if it is available in 2.2 or should I try it from the latest build? Thanks! On Wed, Jun 14, 2017 at 5:32 PM, Micha

Re: What is the real difference between Kafka streaming and Spark Streaming?

This a good question. I really like using Kafka as a centralized source for streaming data in an organization and, with Spark 2.2, we have full support for reading and writing data to/from Kafka in both streaming and batch

Re: What is the real difference between Kafka streaming and Spark Streaming?

June 12, 2017 12:40:37 AM To: Mohammed Guller Cc: vincent gromakowski; yohann jardin; vaquar khan; user Subject: Re: What is the real difference between Kafka streaming and Spark Streaming? Also another difference I see is some thing like Spark Sql where there are logical plans, physical plans,

RE: What is the real difference between Kafka streaming and Spark Streaming?

: Re: What is the real difference between Kafka streaming and Spark Streaming? Also another difference I see is some thing like Spark Sql where there are logical plans, physical plans, Code generation and all those optimizations I don't see them in Kafka Streaming at this time. On Sun, J

Re: What is the real difference between Kafka streaming and Spark Streaming?

than Spark >> Streaming >> >> >> >> Mohammed >> >> >> >> *From:* vincent gromakowski [mailto:vincent.gromakow...@gmail.com] >> *Sent:* Sunday, June 11, 2017 12:09 PM >> *To:* yohann jardin >> *Cc:* kant kodali ; vaquar

Re: What is the real difference between Kafka streaming and Spark Streaming?

latency or throughput? >5. Kafka streaming is relatively new and less mature than Spark >Streaming > > > > Mohammed > > > > *From:* vincent gromakowski [mailto:vincent.gromakow...@gmail.com] > *Sent:* Sunday, June 11, 2017 12:09 PM > *To:* yohann jar

RE: What is the real difference between Kafka streaming and Spark Streaming?

:09 PM To: yohann jardin Cc: kant kodali ; vaquar khan ; user Subject: Re: What is the real difference between Kafka streaming and Spark Streaming? I think Kafka streams is good when the processing of each row is independant from each other (row parsing, data cleaning...) Spark is better when

Re: What is the real difference between Kafka streaming and Spark Streaming?

I think Kafka streams is good when the processing of each row is independant from each other (row parsing, data cleaning...) Spark is better when processing group of rows (group by, ml, window func...) Le 11 juin 2017 8:15 PM, "yohann jardin" a écrit : Hey, Kafka can also do streaming on its own

Re: What is the real difference between Kafka streaming and Spark Streaming?

Hey, Kafka can also do streaming on its own: https://kafka.apache.org/documentation/streams I don’t know much about it unfortunately. I can only repeat what I heard in conferences, saying that one should give a try to Kafka streaming when its whole pipeline is using Kafka. I have no pros/cons t

Re: What is the real difference between Kafka streaming and Spark Streaming?

Hey, Kafka can also do streaming on its own: https://kafka.apache.org/documentation/streams I don't know much about it unfortunately. I can only repeat what I heard in conferences, saying that one should give a try to Kafka streaming when its whole pipeline is using Kafka. I have no pros/cons t

Re: What is the real difference between Kafka streaming and Spark Streaming?

Hi Kant, Kafka is the message broker that using as Producers and Consumers and Spark Streaming is used as the real time processing ,Kafka and Spark Streaming work together not competitors. Spark Streaming is reading data from Kafka and process into micro batching for streaming data, In easy terms