Re: Kafka and Flink integration

>> >> >> So, serialization between producer application -> kafka -> >> flink kafka >> >> consumer will use avro, thrift or kryo right? From there, the >> remaining >> >> pipeline can just use standard pojo serialization,

Re: Kafka and Flink integration

>> -- >> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792p13885.html <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integrat

Re: Kafka and Flink integration

; >> consumer will use avro, thrift or kryo right? From there, the remaining > >> pipeline can just use standard pojo serialization, which would be > better? > >> > >> > >> > >> -- > >> View this message in context: http://apache-flink-user- > mailing-list-arc

Re: Kafka and Flink integration

hich would be better? >> >> >> >> -- >> View this message in context: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792p13885.html >> Sent from the Apache Flink User Mailing List archive. mailing list arc

Re: Kafka and Flink integration

> > Date: 6/21/17 3:10 AM (GMT-08:00) > To: nragon <nuno.goncal...@wedotechnologies.com> > Cc: user@flink.apache.org > Subject: Re: Kafka and Flink integration > > The recommendation has been to avoid Kryo where possible. > > General data exchange: avro o

Re: Kafka and Flink integration

t; > > > -- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792p13885.html > Sent from the Apache Flink User Mailing List archive. mailing list archive at > Nabble.com.

Re: Kafka and Flink integration

-- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792p13885.html > Sent from the Apache Flink User Mailing List archive. mailing list archive at > Nabble.com.

Re: Kafka and Flink integration

g-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792p13885.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Kafka and Flink integration

; > -- > View this message in context: http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink- > integration-tp13792p13882.html > Sent from the Apache Flink User Mailing List archive. mailing list archive > at Nabble.com. >

RE: Kafka and Flink integration

.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792p13882.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

RE: Kafka and Flink integration

Thanks, I'll try to refactor into POJOs. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792p13879.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

RE: Kafka and Flink integration

...@wedotechnologies.com) wrote: Can i have pojo has composition of other pojo? My custom object has many dependencies and in order to refactor it I must also change another 5 classes as well. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink

RE: Kafka and Flink integration

Can i have pojo has composition of other pojo? My custom object has many dependencies and in order to refactor it I must also change another 5 classes as well. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration

RE: Kafka and Flink integration

cal...@wedotechnologies.com> Cc: user@flink.apache.org Subject: Re: Kafka and Flink integration I can only repeat what Gordon wrote on Friday: "It’s usually always recommended to register your classes with Kryo [using registerKryoType()], to avoid the somewhat inefficient classname writing. Also, depending

RE: Kafka and Flink integration

. -Original Message- From: Nico Kruber [mailto:n...@data-artisans.com] Sent: 20 de junho de 2017 16:04 To: user@flink.apache.org Cc: Nuno Rafael Goncalves <nuno.goncal...@wedotechnologies.com> Subject: Re: Kafka and Flink integration No, this is only necessary if you want to re

Re: Kafka and Flink integration

; > > > > > > -Original Message- > From: Nico Kruber [mailto:n...@data-artisans.com] > Sent: 20 de junho de 2017 16:04 > To: user@flink.apache.org > Cc: Nuno Rafael Goncalves <nuno.goncal...@wedotechnologies.com> > Subject: Re: Kafka and Flink int

RE: Kafka and Flink integration

t;nuno.goncal...@wedotechnologies.com> Subject: Re: Kafka and Flink integration No, this is only necessary if you want to register a custom serializer itself [1]. Also, in case you are wondering about registerKryoType() - this is only needed as a performance optimisation. What exactly is your probl

Re: Kafka and Flink integration

hanks > > > > -- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-a > nd-Flink-integration-tp13792p13841.html Sent from the Apache Flink User > Mailing List archive. mailing list archive at Nabble.com. signature.asc Description: This is a digitally signed message part.

Re: Kafka and Flink integration

er-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792p13841.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Kafka and Flink integration

My custom object is used across all job, so it'll be part of checkpoints. Can you point me some references with some examples? -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792p13802.html Sent from

Re: Kafka and Flink integration

objects into kafka and read them with flink. Any tuning advices to use kryo? Such as class registration or something like that? Any examples? Thanks -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink-integration-tp13792

Kafka and Flink integration

I have to produce custom objects into kafka and read them with flink. Any tuning advices to use kryo? Such as class registration or something like that? Any examples? Thanks -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-and-Flink

Re: kafka and flink integration issue

Good to hear. Thanks for letting us know! On Mon, Feb 29, 2016 at 8:14 PM, Pankaj Kumar wrote: > yes versioning was issue . Job is working fine on flink 0.10.2. > > On Mon, Feb 29, 2016 at 3:15 PM, Stephan Ewen wrote: > >> Hi! >> >> A "NoSuchMethodError"

Re: kafka and flink integration issue

yes versioning was issue . Job is working fine on flink 0.10.2. On Mon, Feb 29, 2016 at 3:15 PM, Stephan Ewen wrote: > Hi! > > A "NoSuchMethodError" is always a sign of a version mixup. Please make > sure both versions (cluster and client) are exactly the same. > > Stephan > >

Re: kafka and flink integration issue

Hi! A "NoSuchMethodError" is always a sign of a version mixup. Please make sure both versions (cluster and client) are exactly the same. Stephan On Sat, Feb 27, 2016 at 11:05 AM, Pankaj Kumar wrote: > Yes Robert , > i was trying to start Flink on cluster 0.10.1. > > But

Re: kafka and flink integration issue

Yes Robert , i was trying to start Flink on cluster 0.10.1. But after changing flink version to 0.10.1 , also i am getting the same error. On Sat, Feb 27, 2016 at 2:47 PM, Robert Metzger wrote: > Hi Pankaj, > > I suspect you are trying to start Flink on a cluster with

Re: kafka and flink integration issue

Hi Pankaj, I suspect you are trying to start Flink on a cluster with Flink 0.10.1 installed? On Sat, Feb 27, 2016 at 9:20 AM, Pankaj Kumar wrote: > I am trying to integrate kafka and flink. > my pom file is where {flink.version} is 0.10.2 > > > org.apache.flink >

kafka and flink integration issue

I am trying to integrate kafka and flink. my pom file is where {flink.version} is 0.10.2 org.apache.flink flink-java ${flink.version} org.apache.flink flink-streaming-java ${flink.version} org.apache.flink flink-clients ${flink.version}