Thanks guys. Based on the responses it looks like I need to have a tool of my own to read offsets.
On Mon, Apr 13, 2015 at 3:07 PM, Jiangjie Qin <j...@linkedin.com.invalid> wrote: > Yeah, the current ConsumerOffsetChecker has this issue (maybe bug also) if > the offset storage is Kafka and no offset has been committed. It will > throw ZK exception, which is very confusing. KAFKA-1951 was opened for > this but was not checked in. > > Thanks. > > Jiangjie (Becket) Qin > > On 4/13/15, 9:55 AM, "4mayank" <4may...@gmail.com> wrote: > > >I did a similar change - moved from High Level Consumer to Simple > >Consumer. > >Howerver kafka-consumer-offset-checker.sh throws an exception. Its > >searching the zk path /consumers/<group>/ which does not exist on any of > >my > >zk nodes. > > > >Is there any other tool for getting the offset lag when using Simple > >Consumer? Or am I using kafka-consumer-offset-checker.sh incorrectly for > >Simple Consumer? > > > >Output: > >kafka-consumer-offset-checker.sh --zookeeper 192.168.1.201:2181, > >192.168.1.202:2181,192.168.1.203:2181 --group my-control-group > >Exiting due to: org.apache.zookeeper.KeeperException$NoNodeException: > >KeeperErrorCode = NoNode for /consumers/my-control-group/owners. > > > > > >kafka-consumer-offset-checker.sh --zookeeper 192.168.1.201:2181, > >192.168.1.202:2181,192.168.1.203:2181 --group my-control-group --topic > >my-control > >Exiting due to: org.apache.zookeeper.KeeperException$NoNodeException: > >KeeperErrorCode = NoNode for > >/consumers/my-control-group/offsets/my-control/1. > > > > > >ZK cli output: > > > >./zookeeper-shell.sh 192.168.1.201:2181 > >Connecting to 192.168.1.201:2181 > >Welcome to ZooKeeper! > >JLine support is disabled > > > >WATCHER:: > > > >WatchedEvent state:SyncConnected type:None path:null > >ls /config/topics > >[my-control] > >ls /consumers > >[] > >quit > > > > > >Thanks. > >Mayank. > > > >On Fri, Mar 20, 2015 at 9:54 AM, Jiangjie Qin <j...@linkedin.com.invalid> > >wrote: > > > >> Hi Vamsi, > >> > >> The ConsumerOffsetChecker.scala or kafka-consumer-offset-checker.sh > >>still > >> works. You can use them to check the offsets. > >> If you need to check the offsets programmatically, you can send > >> OffsetsFetcheRequest to broker using simple consumer. You may refer the > >> ConsumerOffsetChecker.scala to see how to find correct broker where > >> corresponding offset manager resides. > >> > >> Jiangjie (Becket) Qin > >> > >> On 3/19/15, 11:54 PM, "Achanta Vamsi Subhash" > >><achanta.va...@flipkart.com> > >> wrote: > >> > >> >Hi, > >> > > >> >We are using 0.8.2.1 currently. > >> > > >> >- How to get the consumer offsets from the offsets topic? > >> >​- Is there any built-in function which I could use? (like in > >> >AdminUtils.scala) > >> >- Is it ok to start a simple consumer and read the offsets from the > >>topic? > >> > > >> >​We used to read the offsets from zookeeper previously for the > >> >HighLevelConsumers. But with the new broker this changed as we are > >>using > >> >Kafka topic for offsets.​ > >> > > >> >-- > >> >Regards > >> >Vamsi Subhash > >> > >> > >
