Just to be clear, what I'm looking for is a way to serialize a POJO class for Kafka but also for Flink, I'm not sure the interface of both frameworks are compatible but it seems they aren't.
For Kafka (producer) I need a Serializer and a Deserializer class, and for Flink (consumer) a SerializationSchema and DeserializationSchema class. Any example of how to put this together would be greatly appreciated. On Thu, Nov 17, 2016 at 9:12 PM, Dromit <dromitl...@gmail.com> wrote: > Tzu-Li Tai, thanks for your response. > > I've seen the example you mentioned before, TaxiRideSchema.java, but it's > way too simplified. > > In a real POJO class you may have multiple fields such as integers, > strings, doubles, etc. So serializing them as a string like in the example > wouldn't work (you can't put together two arbitrary strings and later split > the byte array to get each of them, same for two integers, and nearly any > other types). > > I feel there should be a more general way of doing this regardless of the > fields on the class you're de/serializing. > > What do you do in these cases? It should be a pretty common scenario! > > Regards, > Matt > > On Wed, Nov 16, 2016 at 2:01 PM, Philipp Bussche < > philipp.buss...@gmail.com> wrote: > >> Hi Dromit >> >> I started using Flink with Kafka but am currently looking into Kinesis to >> replace Kafka. >> The reason behind this is that eventually my application will run in >> somebody's cloud and if I go for AWS then I don't have to take care of >> operating Kafka and Zookeeper myself. I understand this can be a >> challenging >> task. >> Up to know where the Kafka bit is only running in a local test >> environment I >> am happy running it as I just start 2 Docker containers and it does the >> job. >> But this also means I have no clue how Kafka really works and what I need >> to >> be careful with. >> Besides knowledge which is required as it seems for Kafka costs is another >> aspect here. >> If one wants to operate a Kafka cluster plus Zookeeper on let's say the >> Amazon cloud this might actually be more expensive than "just" using >> Kinesis >> as a service. >> There are apparently draw backs in terms of functionality and performance >> but for my use case that does not seem to matter. >> >> Philipp >> >> >> >> -- >> View this message in context: http://apache-flink-user-maili >> ng-list-archive.2336050.n4.nabble.com/Why-use-Kafka-after >> -all-tp10112p10155.html >> Sent from the Apache Flink User Mailing List archive. mailing list >> archive at Nabble.com. >> > >
