That is something that you will need to manage manually with the SimpleConsumer. You can store it in ZK (manually) or a file/database/other. You can also store it in Kafka if you send an OffsetCommitRequest (https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommitRequest) but it does involve looking up the "offset manager" so it is slightly more complicated. KAFKA-1013 includes a client that will make that easier.
On Thu, Oct 30, 2014 at 11:32:28AM -0700, Rajiv Kurian wrote: > Does the SimpleConsumer provide any facility to record what offset in a > partition was last processed by a consumer? From the example I have seen > the use of kafka.api.OffsetRequest.EarliestTime() and > kafka.api.OffsetRequest.LatestTime()but nothing that provides consumer -> > Set(partition, offset) > > Is there a facility where the SimpleConsumer will store what the last > offset acknowledged (in ZK like it does for high level consumer) was or is > that something I need to build out of band? > > Thanks, > Rajiv
