Sorry, I forgot the tl;dr on that :)
If you want to know the replication factor for a given partition, you want
to check the length of the replica list in the /brokers/topic/(topic) data
for that partition. Note that all the partitions for a topic do not have to
have the same replication factor (y
Actually, be very careful with this. There are two different things stored
in Zookeeper, and depending on what you're interested in you want to make
sure you're looking at the right one.
If you want to know the replica assignment - that is, what brokers a given
partition is assigned to - you need
Hey, Guozhang,
On Fri, Oct 16, 2015 at 6:20 PM, Guozhang Wang wrote:
> The replica list can be from at /brokers/topics//
> partitions//state
>
Nice, good to know. Thanks! :)
Regards,
Edward​
The replica list can be from at /brokers/topics//
partitions//state
Guozhang
On Fri, Oct 16, 2015 at 2:06 PM, Edward Ribeiro
wrote:
> Umm... the replica *assignment* gets stored under /brokers/topics/
> as "partitions" field, no? Under /brokers/topics//partitions there
> is a znode for each par
Umm... the replica *assignment* gets stored under /brokers/topics/
as "partitions" field, no? Under /brokers/topics//partitions there
is a znode for each partition number with a 'state' as a sub-znode, right?
James, by doing via zkCli.sh:
get /brokers/topics/
You get a json like below:
{"versio
Gwen is right, I was wrong :P
kafka-topics --describe reads the ZK path that Gwen mentioned to return the
replica list, which gives you the idea about replication factor as well.
Thanks,
Guozhang
On Fri, Oct 16, 2015 at 1:51 PM, Gwen Shapira wrote:
> We don't store the replication factor per-s
We don't store the replication factor per-se. When the topic is created, we
use the replication factor to generate replica-assignment, and the replica
assignment gets stored in ZK under: /brokers/topics//partitions/...
This is what gets modified when we re-assign replicas.
Hope this helps.
Gwen
> On Oct 16, 2015, at 1:19 PM, Guozhang Wang wrote:
>
> Replication factor is stored as topic configs that are introduced since
> 0.8.1, you can find it in the wiki you mentioned.
>
Ah, I didn't notice the /config section.
But it still doesn't show the replication factor.
[zk: localhost:2181(C
Umm... Reading the TopicCommand code
https://github.com/apache/kafka/blob/362613347371e9d74184e900ab80ba230940a5c8/core/src/main/scala/kafka/admin/TopicCommand.scala#L192
, it looks like the replication factor (for --describe option, at least) is
calculated by:
1) retrieving the "partitions" map f
Replication factor is stored as topic configs that are introduced since
0.8.1, you can find it in the wiki you mentioned.
Guozhang
On Fri, Oct 16, 2015 at 12:33 PM, James Cheng wrote:
> Hi,
>
> Where is the replication factor for a topic stored? It isn't listed at
> https://cwiki.apache.org/con
Hi,
Where is the replication factor for a topic stored? It isn't listed at
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper.
But the kafka-topics --describe command returns something. Where is it finding
that?
Thanks,
-James
___
11 matches
Mail list logo
