Partition and Replica assignment for a Topic

2014-10-21 Thread Jonathan Creasy
I¹d like to be able to see a little more detail for a topic.

What is the best way to get this information?

Topic   Partition   Replica Broker
topic1  1   1   3
topic1  1   2   4
topic1  1   3   1
topic1  2   1   1
topic1  2   2   3
topic1  2   3   2

I¹d like to be able to create topic allocations dashboards, similar to the
index allocations dashboards in the Elasticsearch plugin Marvell.

Basically, translating index - topic, shard - partition, replica -
replica, node - broker.

-Jonathan




Re: Partition and Replica assignment for a Topic

2014-10-21 Thread Gwen Shapira
Anything missing in the output of:
kafka-topics.sh --describe --zookeeper localhost:2181
?

On Tue, Oct 21, 2014 at 4:29 PM, Jonathan Creasy
jonathan.cre...@turn.com wrote:
 I¹d like to be able to see a little more detail for a topic.

 What is the best way to get this information?

 Topic   Partition   Replica Broker
 topic1  1   1   3
 topic1  1   2   4
 topic1  1   3   1
 topic1  2   1   1
 topic1  2   2   3
 topic1  2   3   2

 I¹d like to be able to create topic allocations dashboards, similar to the
 index allocations dashboards in the Elasticsearch plugin Marvell.

 Basically, translating index - topic, shard - partition, replica -
 replica, node - broker.

 -Jonathan




Re: Partition and Replica assignment for a Topic

2014-10-21 Thread Jonathan Creasy
Heh, I think I was mis-interpreting that output.

Taking this output for example:

Topic:REPL-atl1-us  PartitionCount:256  ReplicationFactor:1
Configs:
Topic: REPL-atl1-us Partition: 0Leader: 32  Replicas:
32Isr: 32
Topic: REPL-atl1-us Partition: 1Leader: 33  Replicas:
33Isr: 33
Topic: REPL-atl1-us Partition: 2Leader: 34  Replicas:
34Isr: 34
Topic: REPL-atl1-us Partition: 3Leader: 35  Replicas:
35Isr: 35
[…]


I read that to mean that partition 0 was primary on broker 32, it had 32
replicas (somewhere) and that there were 32 in-sync replicas.

After you asked I went and looked at the docs on that.

I think it does indeed show me exactly what I’m looking for.

Thanks!



On 10/21/14, 3:32 PM, Gwen Shapira gshap...@cloudera.com wrote:

Anything missing in the output of:
kafka-topics.sh --describe --zookeeper localhost:2181
?

On Tue, Oct 21, 2014 at 4:29 PM, Jonathan Creasy
jonathan.cre...@turn.com wrote:
 I¹d like to be able to see a little more detail for a topic.

 What is the best way to get this information?

 Topic   Partition   Replica Broker
 topic1  1   1   3
 topic1  1   2   4
 topic1  1   3   1
 topic1  2   1   1
 topic1  2   2   3
 topic1  2   3   2

 I¹d like to be able to create topic allocations dashboards, similar to
the
 index allocations dashboards in the Elasticsearch plugin Marvell.

 Basically, translating index - topic, shard - partition, replica -
 replica, node - broker.

 -Jonathan