Thanks, but that's not really what I was looking for. From your answer, I'm guessing you don't have any "Get a batch of messages and stop reading from Kafka" functionality, so I'll just change my strategy to process a batch of messages based on time.
Casey ________________________________________ From: Jun Rao [jun...@gmail.com] Sent: Thursday, December 15, 2011 5:13 PM To: kafka-users@incubator.apache.org Subject: Re: Retrieving unprocessed messages and exiting By default, once started, our consumer (zk-based one) consumes new messages from where it left off last time. After all new messages are consumed and no new messages are produced, the consumer blocks in the message iterator. If you don't want to block in that case, there is a consumer timeout that you can set. Thanks, Jun On Thu, Dec 15, 2011 at 1:49 PM, Sybrandy, Casey < casey.sybra...@six3systems.com> wrote: > Hello, > > Is there a way to retrieve all new messages from a Kafka queue and know > when there we've gotten them all? What I'm looking to do is write a simple > consumer that runs periodically and captures all of the messages that have > been stored in Kafka since it was last ran. > > Btw: I'm doing this in Java and would prefer to only have to interface > with Java classes vs. Scala classes, but I can make due if that's not > possible. > > Casey
