Re: Kafka 0.8.3 - New Consumer - user implemented partition.assignment.strategies?

2015-06-18 Thread Jiangjie Qin
Hi Olof, I am just wondering what is the benefit of rebalancing with minimal number of reassignments here? I am asking this because in new consumer, the rebalance actually is quite cheap on the consumer side - just updating a topic partition set. That means the actually rebalance time on

Re: Kafka 0.8.3 - New Consumer - user implemented partition.assignment.strategies?

2015-06-11 Thread Johansson, Olof
Thanks Guozhang, I agree that it seems to be a common reassignment strategy that should be one of the pre-defined strategies. Do you have a Jira ticket for this specific case, and/or a Jira ticket to add user defined partitions.assignment.strategies that I can watch? / Olof On 10/06/2015 14:35,

Re: Kafka 0.8.3 - New Consumer - user implemented partition.assignment.strategies?

2015-06-10 Thread Guozhang Wang
Hi Olof, Yes we have plans to allow user defined partitions.assignment strategy to pass in to the consumer coordinator; I am not sure if this feature will not be available in the first release of the new consumer in 0.8.3 though. Currently users still have to choose one from the server-defined

Kafka 0.8.3 - New Consumer - user implemented partition.assignment.strategies?

2015-06-10 Thread Johansson, Olof
Is it possible to have a consumer rebalancing partition assignment strategy that will rebalance with a minimal number of reassignments? Per the Kafka 0.9 Consumer Rewrite Design it should be possible to define my own partition assignment strategy: partition.assignment.strategies - may take a