Thanks guys, I got the point. We ended up handling the ser/de in custom wrappers over the generic byte[],byte[] producer to ensure only single producer for the application and neither lose the type safety.
On Wed, May 13, 2015 at 11:31 PM, Guozhang Wang <[email protected]> wrote: > Hello Mohit, > > When we originally design the new producer API we removed the serializer / > deserializer from the old producer and made it generic as accepting only > message<byte[], byte[]>, but we later concluded it would still be more > beneficial to add the serde back into the producer API. And as you observed > one consequence is that if you have multiple data serialization formats > then you have to use one producer for each. > > The recommended way of using Kafka producer in production is to stick with > one serialization format, e.g. Avro, with different topics to have > different schemas that could still be ser-/de- by the single producer. > > Guozhang > > On Wed, May 13, 2015 at 10:03 AM, Mohit Gupta < > [email protected] > > wrote: > > > Hello, > > > > I've a question regarding the design of the new Producer API. > > > > As per the design (KafkaProducer<K,V>), it seems that a separate producer > > is required for every combination of key and value type. Where as, in > > documentation ( and elsewhere ) it's recommended to create a single > > producer instance per application and share it among the all the threads > > for best performance? > > > > One way to create only single producer would be to use byte[] as > key/value > > type and handle the serialization at the client itself, rather than the > > producer, similar to the example in javadocs. But wouldn't this defeat > the > > purpose of using generics in the producer? > > > > Specific to our use case, we have multiple types of messages, where each > > message type can have multiple custom serializers. And, a message can be > > pushed into mulitple topics with different serialization. > > > > > > -- > > Best Regards, > > > > Mohit Gupta > > > > > > -- > -- Guozhang > -- Best Regards, Mohit Gupta
