Re: New consumer - poll/seek javadoc confusing, need clarification

2015-07-23 Thread Jason Gustafson
Hey Stevo, Thanks for the early testing on the new consumer! This might be a bug. I wonder if it could also be explained by partition rebalancing. In the current implementation, a rebalance will clear the old positions (including those that were seeked to). I think it's debatable whether this

Re: New consumer - poll/seek javadoc confusing, need clarification

2015-07-23 Thread Stevo Slavić
Thanks Ewen for heads up. It's great that seek is not needed in between poll when business goes as usual. In edge case, when my logic detects it needs to go back and reread events from given position in history, I use seek. I found out that next poll after seek will not respect offset used in

Re: New consumer - poll/seek javadoc confusing, need clarification

2015-07-23 Thread Stevo Slavić
Strange, if after seek I make several poll requests, eventually it will read/return messages from offset that seek set. On Thu, Jul 23, 2015 at 11:03 AM, Stevo Slavić ssla...@gmail.com wrote: Thanks Ewen for heads up. It's great that seek is not needed in between poll when business goes as

Re: New consumer - poll/seek javadoc confusing, need clarification

2015-07-21 Thread Ewen Cheslack-Postava
On Tue, Jul 21, 2015 at 2:38 AM, Stevo Slavić ssla...@gmail.com wrote: Hello Apache Kafka community, I find new consumer poll/seek javadoc a bit confusing. Just by reading docs I'm not sure what the outcome will be, what is expected in following scenario: - kafkaConsumer is instantiated

Re: New consumer - poll/seek javadoc confusing, need clarification

2015-07-21 Thread James Cheng
On Jul 21, 2015, at 9:15 AM, Ewen Cheslack-Postava e...@confluent.io wrote: On Tue, Jul 21, 2015 at 2:38 AM, Stevo Slavić ssla...@gmail.com wrote: Hello Apache Kafka community, I find new consumer poll/seek javadoc a bit confusing. Just by reading docs I'm not sure what the outcome

New consumer - poll/seek javadoc confusing, need clarification

2015-07-21 Thread Stevo Slavić
Hello Apache Kafka community, I find new consumer poll/seek javadoc a bit confusing. Just by reading docs I'm not sure what the outcome will be, what is expected in following scenario: - kafkaConsumer is instantiated with auto-commit off - kafkaConsumer.subscribe(someTopic) -