By default, once started, our consumer (zk-based one) consumes new messages from where it left off last time. After all new messages are consumed and no new messages are produced, the consumer blocks in the message iterator. If you don't want to block in that case, there is a consumer timeout that you can set.
Thanks, Jun On Thu, Dec 15, 2011 at 1:49 PM, Sybrandy, Casey < casey.sybra...@six3systems.com> wrote: > Hello, > > Is there a way to retrieve all new messages from a Kafka queue and know > when there we've gotten them all? What I'm looking to do is write a simple > consumer that runs periodically and captures all of the messages that have > been stored in Kafka since it was last ran. > > Btw: I'm doing this in Java and would prefer to only have to interface > with Java classes vs. Scala classes, but I can make due if that's not > possible. > > Casey
