Re: Producer offset commit API

2016-05-16 Thread Kanagha
Thanks for providing the links. I 'll test it out using offsetStorageReader. Kanagha On Mon, May 16, 2016 at 10:09 AM, Christian Posta wrote: > If you're using KafkaConnect, it does it for you! > > basically you set the sourceRecord's "sourcePartition" and

Re: Producer offset commit API

2016-05-16 Thread Christian Posta
If you're using KafkaConnect, it does it for you! basically you set the sourceRecord's "sourcePartition" and "sourceOffset" fields (

Re: Producer offset commit API

2016-05-16 Thread Tom Crayford
Hi, Producers don't track offsets in the same way, so there is no producer offset API. Thanks Tom Crayford Heroku Kafka On Mon, May 16, 2016 at 5:25 PM, Kanagha wrote: > Hi, > > I am trying to find out the API for committing producer offset for Kafka > > I found this

Producer offset commit API

2016-05-16 Thread Kanagha
Hi, I am trying to find out the API for committing producer offset for Kafka I found this example: https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka Which would work for committing consumer offsets. Is there a separate API for committing

Re: offset commit api

2014-09-14 Thread Neha Narkhede
Koshy jjkosh...@gmail.com wrote: If you are using the high-level consumer then yes you would be under a group. If you are using the simple consumer then no. I just saw your earlier email in this thread - if you are using the simple consumer you can still use the offset commit API and set

Re: offset commit api

2014-08-18 Thread Weide Zhang
Thanks Joel. Do you know if I'm using consumer group string On Mon, Aug 4, 2014 at 3:16 PM, Joel Koshy jjkosh...@gmail.com wrote: Weide, 0.8.1.1 does not support offsets storage in Kafka. The brokers do support offset commit requests/fetches but simply forward to ZooKeeper - you can issue

Re: offset commit api

2014-08-18 Thread Joel Koshy
If you are using the high-level consumer then yes you would be under a group. If you are using the simple consumer then no. I just saw your earlier email in this thread - if you are using the simple consumer you can still use the offset commit API and set a group manually. On Mon, Aug 18, 2014

Re: offset commit api

2014-08-04 Thread Weide Zhang
Hi It seems to me that 0.8.1.1 doesn't have the ConsumerMetadata API. So what broker I should choose in order to commit and fetch offset information ? Shall I use zookeeper for offset to manage it manually instead ? Thanks, Weide On Sun, Aug 3, 2014 at 4:34 PM, Weide Zhang weo...@gmail.com