Hello! >> Whether you commit offsets to Kafka itself (stored in offsets topic) or ZK >> or both depends on the settings in two properties: offset.storage and >> >>dual.commit.enabled (here >> <http://kafka.apache.org/documentation.html#consumerconfigs>). Currently ZK >> is the default.
Thanks for explanation.... but I have more questions... 1. As far as I understand when I use offset.storage=kafka and dual.commit.enabled in consumer (I mean java-based) then when I call commit method commit then internally client commit offset to kafka and then to zk.... Is it correct? 2. if first is correct ...then what happened if client crash between this 2 operations (commit 2 kafka and commit 2 zookeeper)? 3. What happened if broker crash.... And my client support only zookeeper? (I guess that I read message from beggining) Is it correct? Or broker explicitly sync kafka commit with zookeeper when started? Thanks in advance, Aleksey -----Original Message----- From: Ben Stopford [mailto:b...@confluent.io] Sent: Friday, October 9, 2015 5:54 PM To: users@kafka.apache.org Subject: [!!Mass Mail]Re: Dual commit with zookeeper and kafka Hi Alexey Whether you commit offsets to Kafka itself (stored in offsets topic) or ZK or both depends on the settings in two properties: offset.storage and dual.commit.enabled (here <http://kafka.apache.org/documentation.html#consumerconfigs>). Currently ZK is the default. Commits happen either periodically (auto.commit.enable=true && auto.commit.interval.ms) or explicitly. For example the high level consumer has a commitOffsets method. I expect the python bindings have a similar function. The rest proxy offers this function too. Hope that helps B > On 6 Oct 2015, at 23:06, Рябков Алексей Николаевич <a.ryab...@ntc-vulkan.ru> > wrote: > > Hello! > > Can somebody explain me how to use multiple consumers with different commit > storage... > For example, java-based consumers use kafka commit storage... > python-based consumers use zookeeper commit storage > > My question is: > Is it true that when one consumer commit to kafka, server also commit > automatically to zookeeper (so all consumers see the same commit) Is > it true that when one consumer commit to zookeeper, server also > commit automatically to kafka (so all consumers see the same commit) > Is it true that after server restart it read commits from kafka > storage and publish (sync) commit to zookeeper (so after server > restart all consumers can see also the same commit) > > Thanks in advance, Aleksey
