Hey Tom, Yes, it is possible that the poll() will rebalance and resume fetching for a previously paused partition. First thought is to use a ConsumerRebalanceListener to re-pause the partitions after the rebalance completes.The rebalance listener offers two hooks: onPartitionsRevoked() is called before the rebalance starts, and onPartitionsAssigned() after it finishes. If you you use onPartitionsAssigned(), you can re-pause before the consumer begins fetching again. Note that you'll have to handle the case that the partition assignment has changed.
Not sure I understand the third question, but seeks should be to the next offset you want to read. -Jason On Thu, Jan 28, 2016 at 8:53 AM, Tom Dearman <tom.dear...@gmail.com> wrote: > I am trying to enable throttling on a consumer poller. My understanding > of the consumer is that it has a single threaded design so that all comms > affecting the consumer are sent during a client threads call to poll and > commitSynch. We have therefor designed our throttling by using the > consumer pause, then continuing to poll for some duration, and then calling > resume. > > 1. Is it the case that during that poll, a rebalance could occur and that > since subscription state is not maintained across consumer balances, when > the rest of the poll(s) code runs (as the assignment check is done first) > this call could result in records being returned from the poll? > 2. If so how can I get around this, is there a better way to implement a > throttle? > 3. If I implement throttling in the way I am doing can I reset the offset > using the smallest offset returned for each TopicPartition and seek? If I > do use seek, do I pass in the smallest value per TopicPartition or is it > that value minus 1? > > Thanks, > > Tom
