Re: are offsets per consumer or per consumer group?

2018-02-08 Thread Xavier Noria
Thanks very much Luke!

Re: are offsets per consumer or per consumer group?

2018-02-08 Thread Luke Steensen
1) In more recent versions of Kafka, the consumer group coordinator runs on the broker. Previously, there was a "high level consumer" that spoke directly to zookeeper and did group management within the client libraries, but this is no longer used. 2) That depends on when your consumer commits

Re: are offsets per consumer or per consumer group?

2018-02-08 Thread Xavier Noria
On Thu, Feb 8, 2018 at 4:27 PM, Luke Steensen < luke.steen...@braintreepayments.com> wrote: Offsets are maintained per consumer group. When an individual consumer > crashes, the consumer group coordinator will detect that failure and > trigger a rebalance. This redistributes the partitions being

Re: are offsets per consumer or per consumer group?

2018-02-08 Thread Luke Steensen
Offsets are maintained per consumer group. When an individual consumer crashes, the consumer group coordinator will detect that failure and trigger a rebalance. This redistributes the partitions being consumed across the available consumer processes, using the most recently committed offset for