This deteriorated somewhat rapidly. First it was using only 3 brokers (of the five).. then 2.. finally only one broker/thread was doing all the work.
Then the topic/group stopped responding at all. Now if I call 'kafka-consumer-groups <bootstrap> <topic> --new-consumer --describe' all it says is 'Consumer group `somegroup` is rebalancing.' It's been doing this for about 30 minutes now so Im assuming it's completely wedged. Looks like other consumers are still running. What did I do wrong here? How to fix it? I've stopped all the consumers. No change. Kafka 0.10.1.0 On Fri, Dec 2, 2016 at 9:58 AM, Jon Yeargers <jon.yearg...@cedexis.com> wrote: > I have a topic with 10 partitions. I have 10 consumers (5 processes x 2 > threads) on 5 separate machines. Seem like a decent match? > > So why does kafka rebalance and commonly assign two (or more) partitions > to a single thread? This leaves threads idling and the partitions start > lagging. > > Is there some way to 'smarten up' this process? > > Better strategy for sharing out work between consumers? >
