Re: Known issues or points of failure during consumer rebalance

The most severe issue I've run into is a poorly timed GC pause can actually lead to a situation where rebalancing leaves a partition completely un-owned. It's important to make sure that rebalance.max.retries * rebalance.backoff.ms is longer than any GC pause that your consumers experience. A

Consumer with more stable partition assignments?

One of the big issues we run into with the 0.8 high-level consumer is the instability of partition assignments during a rebalance. The simple lexicographic assignment strategy means that if one consumer instance dies (and potentially a new instance with a different consumerId gets spun up to

Re: Multiple consumer groups with same group id on a single topic

On Mon, Mar 9, 2015 at 10:38 AM, Phill Tomlinson philltomlin...@fico.com wrote: Hi, I have two separate consumer groups on different JVM processes, but both have the same group.id. You've said this twice, and I think it's creating some confusion, because the group.id is exactly what

Is it actually a bad idea to set 'consumer.id' explicitly?

https://github.com/apache/kafka/blob/0.8.2/core/src/main/scala/kafka/consumer/ConsumerConfig.scala#L101 suggests that 'consumer.id' should only be set explicitly for testing purposes. Is there a reason that it would be a bad idea to set it ourselves for production use? The reason I am asking is