Thanks for sharing your use case! In the new consumer APIs, we intend to support custom offset management as well as custom partition assignment. We have a thread discussing the new consumer API javadoc to collect feedback from users. It will greatly help to have your feedback as well - http://grokbase.com/t/kafka/users/142avhm32j/new-consumer-api-discussion
Thanks, Neha On Tue, Feb 11, 2014 at 11:25 AM, Pete Matern <p...@jivesoftware.com> wrote: > Hi - > Recently Mr. Kreps was kind enough to answer a bunch of questions from > some of us here at Jive software. One subject that particularly caught our > interest was the new java client, which sounds like it will super useful. > > We have some client code which sits on top of SimpleConsumer which we use > for cases where we want to manage our own offsets, but also for cases where > we don't care about persistent offsets at all, and just want to consume > from the tip of a given partition at the point in time when the consuming > process spins up. I've attached the source in case it might be useful to > see what we are doing. Thank you, and I hope it's helpful. > > -pete
