Re: Partition ownership with high-level consumer

2015-10-07 Thread Ewen Cheslack-Postava
And you can get the current assignment in the new consumer after the rebalance completes too: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L593 On Tue, Oct 6, 2015 at 5:27 PM, Gwen Shapira wrote: > Ah,

Partition ownership with high-level consumer

2015-10-06 Thread Joey Echeverria
Hi! Is there a way to track current partition ownership when using the high-level consumer? It looks like the rebalance callback only tells me the partitions I'm (potentially) losing. -Joey

Re: Partition ownership with high-level consumer

2015-10-06 Thread Gwen Shapira
Zookeeper will have this information under /consumers//owners On Tue, Oct 6, 2015 at 12:22 PM, Joey Echeverria wrote: > Hi! > > Is there a way to track current partition ownership when using the > high-level consumer? It looks like the rebalance callback only tells me the >

Re: Partition ownership with high-level consumer

2015-10-06 Thread Joey Echeverria
But nothing in the API? -Joey On Tue, Oct 6, 2015 at 3:43 PM, Gwen Shapira wrote: > Zookeeper will have this information under /consumers//owners > > > > On Tue, Oct 6, 2015 at 12:22 PM, Joey Echeverria wrote: > > > Hi! > > > > Is there a way to track

Re: Partition ownership with high-level consumer

2015-10-06 Thread Gwen Shapira
I don't think so. AFAIK, even the new API won't send this information to every consumer, because in some cases it can be huge. On Tue, Oct 6, 2015 at 1:44 PM, Joey Echeverria wrote: > But nothing in the API? > > -Joey > > On Tue, Oct 6, 2015 at 3:43 PM, Gwen Shapira

Re: Partition ownership with high-level consumer

2015-10-06 Thread Joey Echeverria
I really only want them for the partitions I own. The client should know that in order to acquire the zookeeper locks and could potentially execute a callback to tell me the partitions I own after a rebalance. -Joey On Tue, Oct 6, 2015 at 4:08 PM, Gwen Shapira wrote: > I