Re: commitOffsets by partition 0.8-beta

2014-04-23 Thread Jun Rao
ckPointedOffset.put(key,nextOffset); > } > } > > Can you please review this ?. > > > > -Original Message----- > From: Seshadri, Balaji > Sent: Wednesday, April 23, 2014 12:01 PM > To: 'users@kafka.apache.org' > Subject: RE: commitOffsets by

RE: commitOffsets by partition 0.8-beta

2014-04-23 Thread Seshadri, Balaji
:01 PM To: 'users@kafka.apache.org' Subject: RE: commitOffsets by partition 0.8-beta Does check point avoid duplicate update to zookeeper ?. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Wednesday, April 23, 2014 10:14 AM To: users@kafka.apache.org Subject: Re: co

RE: commitOffsets by partition 0.8-beta

2014-04-23 Thread Seshadri, Balaji
Does check point avoid duplicate update to zookeeper ?. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Wednesday, April 23, 2014 10:14 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta Take a look at the example in https://cwiki.apache.org

RE: commitOffsets by partition 0.8-beta

2014-04-23 Thread Seshadri, Balaji
] Sent: Wednesday, April 23, 2014 10:14 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta Take a look at the example in https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example Thanks, Jun On Wed, Apr 23, 2014 at 9:01 AM, Seshadri, Balaji

Re: commitOffsets by partition 0.8-beta

2014-04-23 Thread Ian Friedman
PM, Seshadri, Balaji wrote: > I'm not seeing that API in java MessageAndMeta,is this part of > ConsumerIterator. > > > -Original Message- > From: Jun Rao [mailto:jun...@gmail.com] > Sent: Wednesday, April 23, 2014 8:47 AM > To: users@kafka.apache.org &

Re: commitOffsets by partition 0.8-beta

2014-04-23 Thread Jun Rao
Original Message- > From: Jun Rao [mailto:jun...@gmail.com] > Sent: Wednesday, April 23, 2014 8:47 AM > To: users@kafka.apache.org > Subject: Re: commitOffsets by partition 0.8-beta > > The checkpointed offset should be the offset of the next message to be > consu

RE: commitOffsets by partition 0.8-beta

2014-04-23 Thread Seshadri, Balaji
I'm not seeing that API in java MessageAndMeta,is this part of ConsumerIterator. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Wednesday, April 23, 2014 8:47 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta The checkpointed offset s

Re: commitOffsets by partition 0.8-beta

2014-04-23 Thread Jun Rao
consumed > offset. > > -Original Message- > From: Jun Rao [mailto:jun...@gmail.com] > Sent: Tuesday, April 22, 2014 9:52 PM > To: users@kafka.apache.org > Subject: Re: commitOffsets by partition 0.8-beta > > Do you have auto commit disabled? > > Thanks, >

RE: commitOffsets by partition 0.8-beta

2014-04-22 Thread Seshadri, Balaji
Yes I disabled it. My doubt is the path should have offset to be consumed or last consumed offset. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Tuesday, April 22, 2014 9:52 PM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta Do you have

Re: commitOffsets by partition 0.8-beta

2014-04-22 Thread Jun Rao
zookeeper path it has 6 after sometimes. > > Does any other process updates it ?. > > > From: Seshadri, Balaji > Sent: Friday, April 18, 2014 11:50 AM > To: 'users@kafka.apache.org' > Subject: RE: commitOffsets by partition 0

RE: commitOffsets by partition 0.8-beta

2014-04-22 Thread Seshadri, Balaji
uot;+metaData.getPartitionNumber(),metaData.getOffSet()+""); } Thanks, Balaji -Original Message- From: Seshadri, Balaji [mailto:balaji.sesha...@dish.com] Sent: Tuesday, April 22, 2014 8:10 PM To: 'users@kafka.apache.org' Subject: RE: commitOffsets by partition 0.8-beta I'm

RE: commitOffsets by partition 0.8-beta

2014-04-22 Thread Seshadri, Balaji
From: Seshadri, Balaji Sent: Friday, April 18, 2014 11:50 AM To: 'users@kafka.apache.org' Subject: RE: commitOffsets by partition 0.8-beta Thanks Jun. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Friday, April 18, 2014 11:37 AM To: users@kafka.apache.

RE: commitOffsets by partition 0.8-beta

2014-04-18 Thread Seshadri, Balaji
Thanks Jun. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Friday, April 18, 2014 11:37 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta We don't have the ability to commit offset at the partition level now. This feature probably

Re: commitOffsets by partition 0.8-beta

2014-04-18 Thread Jun Rao
We don't have the ability to commit offset at the partition level now. This feature probably won't be available until we are done with the consumer rewrite, which is 3-4 months away. If you want to do sth now and don't want to use SimpleConsumer, another hacky way is to turn off auto offset commit