Re: Offset Management in Kafka
This might help : https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka http://www.slideshare.net/jjkoshy/offset-management-in-kafka Thanks, Mayuresh On Thu, Oct 15, 2015 at 5:39 AM, Kiran Singh wrote: > Hi > > I am looking for offset management in Kafka. > Is there any link which can explain how offset are updated in Kafka once a > consumer read a message. > -- -Regards, Mayuresh R. Gharat (862) 250-7125
Offset Management in Kafka
Hi I am looking for offset management in Kafka. Is there any link which can explain how offset are updated in Kafka once a consumer read a message.
Re: offset-management-in-kafka
You are looking at the newer, and not-ready-yet consumer there. The configuration you are looking for is defined here: core/src/main/scala/kafka/consumer/ConsumerConfig.scala Gwen On Wed, Apr 8, 2015 at 9:25 AM, Yosi Botzer wrote: > Hi, > > I am looking at the java > class org.apache.kafka.clients.consumer.ConsumerConfig and I do not see > there a constant for offsets.storage > > Am I missing something? > > This is my pom dependency definition: > > > org.apache.kafka > kafka_2.10 > 0.8.2.1 > > > On Wed, Apr 8, 2015 at 6:29 PM, Gwen Shapira > wrote: > > > This is available from 0.8.2.0, and is enabled on server by default. The > > consumer needs to specify offsets.storage parameter - the default is > still > > zookeeper, so the consumers should set it to 'kafka'. > > > > The documentation also explain how to migrate from zookeeper offsets to > > kafka offsets. > > > > Gwen > > > > On Wed, Apr 8, 2015 at 8:13 AM, Yosi Botzer > wrote: > > > > > Hi, > > > > > > I have seen this presentation: > > > http://www.slideshare.net/jjkoshy/offset-management-in-kafka > > > > > > describing a new approach for offset management. > > > > > > I wanted to know from which kafka version this option will be available > > and > > > will it be available by default > > > > > > Thanks > > > Yosi > > > > > >
Re: offset-management-in-kafka
Hi, I am looking at the java class org.apache.kafka.clients.consumer.ConsumerConfig and I do not see there a constant for offsets.storage Am I missing something? This is my pom dependency definition: org.apache.kafka kafka_2.10 0.8.2.1 On Wed, Apr 8, 2015 at 6:29 PM, Gwen Shapira wrote: > This is available from 0.8.2.0, and is enabled on server by default. The > consumer needs to specify offsets.storage parameter - the default is still > zookeeper, so the consumers should set it to 'kafka'. > > The documentation also explain how to migrate from zookeeper offsets to > kafka offsets. > > Gwen > > On Wed, Apr 8, 2015 at 8:13 AM, Yosi Botzer wrote: > > > Hi, > > > > I have seen this presentation: > > http://www.slideshare.net/jjkoshy/offset-management-in-kafka > > > > describing a new approach for offset management. > > > > I wanted to know from which kafka version this option will be available > and > > will it be available by default > > > > Thanks > > Yosi > > >
Re: offset-management-in-kafka
Thanks Gwan, Are there any plans to change the default from zookeeper to kafka in the future? On Wed, Apr 8, 2015 at 6:29 PM, Gwen Shapira wrote: > This is available from 0.8.2.0, and is enabled on server by default. The > consumer needs to specify offsets.storage parameter - the default is still > zookeeper, so the consumers should set it to 'kafka'. > > The documentation also explain how to migrate from zookeeper offsets to > kafka offsets. > > Gwen > > On Wed, Apr 8, 2015 at 8:13 AM, Yosi Botzer wrote: > > > Hi, > > > > I have seen this presentation: > > http://www.slideshare.net/jjkoshy/offset-management-in-kafka > > > > describing a new approach for offset management. > > > > I wanted to know from which kafka version this option will be available > and > > will it be available by default > > > > Thanks > > Yosi > > >
Re: offset-management-in-kafka
This is available from 0.8.2.0, and is enabled on server by default. The consumer needs to specify offsets.storage parameter - the default is still zookeeper, so the consumers should set it to 'kafka'. The documentation also explain how to migrate from zookeeper offsets to kafka offsets. Gwen On Wed, Apr 8, 2015 at 8:13 AM, Yosi Botzer wrote: > Hi, > > I have seen this presentation: > http://www.slideshare.net/jjkoshy/offset-management-in-kafka > > describing a new approach for offset management. > > I wanted to know from which kafka version this option will be available and > will it be available by default > > Thanks > Yosi >
offset-management-in-kafka
Hi, I have seen this presentation: http://www.slideshare.net/jjkoshy/offset-management-in-kafka describing a new approach for offset management. I wanted to know from which kafka version this option will be available and will it be available by default Thanks Yosi
