Re: records with key as string and value as java ArrayList in topic

2020-05-12 Thread Pushkar Deole
Thanks Liamessentially, it would be an internal topic that we would be creating to use as a cache store by accessing topic through a GlobalKTable, so the problem you mentioned above for storing Hashmap may not apply there On Tue, May 12, 2020, 21:25 Liam Clarke-Hutchinson <

Re: records with key as string and value as java ArrayList in topic

2020-05-12 Thread Liam Clarke-Hutchinson
Hi Pushkar, Just wanted to say, as someone with battle scars from ActiveMQ and Camel, there's very many good reasons to avoid Java serialization on a messaging system. What if you need to tail a topic from the console? What if your testers want to access in their pytests? Etc. And that's not even

Re: records with key as string and value as java ArrayList in topic

2020-05-11 Thread Pushkar Deole
And by the way, confluent has provided KafkaAvroSerializer/Deserialier. Can't they be used to do conversion for java types? On Tue, May 12, 2020 at 10:09 AM Pushkar Deole wrote: > Ok... so jackson json serialization is the way to go for hashmaps as well? > > On Mon, May 11, 2020 at 7:57 PM John

Re: records with key as string and value as java ArrayList in topic

2020-05-11 Thread Pushkar Deole
Ok... so jackson json serialization is the way to go for hashmaps as well? On Mon, May 11, 2020 at 7:57 PM John Roesler wrote: > Oh, my mistake. I thought this was a different thread :) > > You might want to check, but I don’t think there is a kip for a map serde. > Of course, you’re welcome to

Re: records with key as string and value as java ArrayList in topic

2020-05-11 Thread John Roesler
Oh, my mistake. I thought this was a different thread :) You might want to check, but I don’t think there is a kip for a map serde. Of course, you’re welcome to start one. Thanks, John On Mon, May 11, 2020, at 09:14, John Roesler wrote: > Hi Pushkar, > > I don’t think there is. You’re

Re: records with key as string and value as java ArrayList in topic

2020-05-11 Thread John Roesler
Hi Pushkar, I don’t think there is. You’re welcome to start one if you think it would be a useful addition. Before worrying about it further, though, you might want to check the InMemoryKeyValueStore implementation, since my answer was from memory. Thanks, John On Mon, May 11, 2020, at

Re: records with key as string and value as java ArrayList in topic

2020-05-11 Thread Pushkar Deole
John, is there KIP in progress for supporting Java HashMap also? On Sun, May 10, 2020, 00:47 John Roesler wrote: > Yes, that’s correct. It’s only for serializing the java type ‘byte[]’. > > On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote: > > Thanks John... I got to finish the work in few

Re: records with key as string and value as java ArrayList in topic

2020-05-09 Thread John Roesler
Yes, that’s correct. It’s only for serializing the java type ‘byte[]’. On Thu, May 7, 2020, at 10:37, Pushkar Deole wrote: > Thanks John... I got to finish the work in few days so need to get it > quick, so looking for something ready. I will take a look at jackson json. > > By the way, what is

Re: records with key as string and value as java ArrayList in topic

2020-05-07 Thread Pushkar Deole
Thanks John... I got to finish the work in few days so need to get it quick, so looking for something ready. I will take a look at jackson json. By the way, what is the byteArrayserializer? As the name suggests, it is for byte arrays so won't work for java ArrayList, right? On Thu, May 7, 2020

Re: records with key as string and value as java ArrayList in topic

2020-05-07 Thread John Roesler
Hi Pushkar, If you’re not too concerned about compactness, I think Jackson json serialization is the easiest way to serialize complex types. There’s also a kip in progress to add a list serde. You might take a look at that proposal for ideas to write your own. Thanks, John On Thu, May 7,

Re: records with key as string and value as java ArrayList in topic

2020-05-07 Thread Nicolas Carlot
Won't say it's a good idea to use java serialized classes for messages, but you should use a byteArraySerializer if you want to do such things Le jeu. 7 mai 2020 à 14:32, Pushkar Deole a écrit : > Hi All, > > I have a requirement to store a record with key as java String and value as > java's

records with key as string and value as java ArrayList in topic

2020-05-07 Thread Pushkar Deole
Hi All, I have a requirement to store a record with key as java String and value as java's ArrayList in the kafka topic. Kafka has by default provided a StringSerializer and StringDeserializer, however for java ArrayList, how can get serializer. Do I need to write my own? Can someone share if