W.r.t. item 3, it may be an issue of whether the group is active or not and whether it has been flushed from the memory cache. Some discussion here: http://grokbase.com/t/kafka/users/15bsxxgp49/new-kafka-consumer-groups-sh-not-showing-inactive-consumer-gorup
Ram On Thu, Dec 8, 2016 at 8:36 AM, Arvindan Thulasinathan < aravindan.thulasinat...@oracle.com> wrote: > Hi, > I am using “org.apache.apex.malhar.kafka.AbstractKafkaInputOperator” > which supports 0.9.0 + version of Kafka consumer API. I have InitialOffset > set to “APPLICATION_OR_LATEST” > > I have a few questions. > 1. When I kill my application and restart, I expect the app to restart > from the last offset it consumed. But this is not happening. It > automatically starts consuming from the latest offest. Is it because I am > killing the app and not shutting-down? > 2. Is there a way to measure my Kafka consumption rate from the Apex > application? I am looking for something out-of-box rather than sending the > consumption rate over a stream to an output operator. > 3. I couldn’t track offsets on the Kafka broker. > I tried using: "bin/kafka-consumer-groups.sh --new-consumer > --bootstrap-server <kafa-broker>:<port> —list” script on my Kafka broker. > But the result is empty. Any idea, if I need to add some parameter/config > to my Apex Kafka consumer to enable it to commit offset to Broker? > > > Thanks, > Aravindan >
