Yes, our service do handle the consumer group offsets by itself. It seems to be that the negative value happens because all replicas of a partition are dead, and Kafka create a new replica for the partition, and the high level consumer is not aware of the Kafka server side change.
On Sun, Oct 19, 2014 at 8:06 PM, Jun Rao <jun...@gmail.com> wrote: > Haven't seen this one before. Are you manually committing offsets yourself? > > Thanks, > > Jun > > On Fri, Oct 17, 2014 at 11:10 PM, Yu Yang <yuyan...@gmail.com> wrote: > > > Hi, > > > > I have a Kafka 0.8.1 cluster. I used the ConsumerOffsetChecker tool to > > check the lag of consumer groups. I found that for some partition, the > tool > > returns negative value for the "lag " column. Is this a known issue that > > has been seen before? I find that the negative value prevents the > consumer > > consuming the latest events in these partitions. How can we work around > > the problem? > > > > The following is the command: > > > > ~/kafka_2.9.2-0.8.1$ bin/kafka-run-class.sh > > kafka.tools.ConsumerOffsetChecker --group topic_partition --zkconnect > > zk001:2181 --topic the_topic | grep ' -' > > > > The following is part of the output. The topic that I am checking has 128 > > partitions, and the tool returns negative value for 63 partitions. > > > > topic_partition event 6 202936733 > > 28822327 * -*174114406 topic_partition_m031_29714_20-0 > > > > topic_partition event 10 177322216 > > 36578944 * -*140743272 topic_partition_m032_16773_16-0 > > > > topic_partition event 11 187891640 > > 28999350 * -*158892290 topic_partition_m032_16773_17-0 > > > > > > > > Thanks! > > > > -Yu > > >
