Re: Kafka Producer Partition Key Selection

2018-08-29 Thread Gaurav Bajaj
Hello Senthil, In our case we use NULL as message Key to achieve even distribution in producer. With that we were able to achieve very even distribution with that. Our Kafka client version is 0.10.1.0 and Kafka broker version is 1.1 Thanks, Gaurav On Wed, Aug 29, 2018 at 9:15 AM, SenthilKumar K

Re: Kafka Producer Partition Key Selection

2018-08-29 Thread M. Manna
Why can't we override the DefaultPartitioner, and simply override paritition() method, such that it will redistribute to all partitions in round robin fashion. Round-Robin partitioner and StickyAssignor (consumer) should work nicely for any publish subscribe system. On Wed, 29 Aug 2018 at 09:39,

Re: Kafka Producer Partition Key Selection

2018-08-29 Thread SenthilKumar K
Thanks Gaurav. Did you notice side effect mentioned in this page : https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified ? --Senthil On Wed, Aug 29, 2018 at 2:02 PM Gaurav Bajaj wrote: > Hello Senthil, > > In o