Largest works - however, autooffset only applies if the consumer group is new (non-existent in ZK) or if the stored offset is out of range. If you restart your consumer at a later point it will resume from the last check-pointed offset. So from the scenario you describe (down for a period of time...) it appears that it wouldn't be enough for you. Depending on the details of your set up, one thing you could do is to clean up the zookeeper entries for your consumer group before bringing up your consumer service again.
Joel On Mon, Jan 23, 2012 at 4:49 PM, Evan Chan <e...@ooyala.com> wrote: > Michael, > > It seems there is a property "autooffset.reset" here: > http://incubator.apache.org/kafka/configuration.html > > which you can set to "largest". I haven't used it though so I can't be > sure. > > -Evan > > > On Mon, Jan 23, 2012 at 4:12 PM, Michael Luban <michael.lu...@gmail.com>wrote: > >> I'm interested in having my consumer service start consuming the most >> recent message rather than the message corresponding to the last offset >> stored. For instance, if the kafka consumer service is down for a period >> of time, I would like to skip over all messages since the last recorded >> offset and begin consuming from the most recent message. >> >> According to the following, it seems I have to use SimpleConsumer to >> achieve this? Seems like this would be difficult to manage in a "load >> balanced" consumer scenario (federation of consumers in the same group >> consuming the same topic). >> >> https://groups.google.com/group/kafka-dev/browse_thread/thread/ee7fe5f8eef49a4d/a4469233880a7f76?lnk=gst&q=offset#a4469233880a7f76 >> >> Any additional insight appreciated. >> > > > > -- > -- > *Evan Chan* > Senior Software Engineer | > e...@ooyala.com | (650) 996-4600 > www.ooyala.com | blog <http://www.ooyala.com/blog> | > @ooyala<http://www.twitter.com/ooyala>
