I have used the kafka-topics.sh script to create a topic with a number of partitions.
bin/kafka-topics.sh --zookeeper myZKPath-create --topic myTopic --partitions 1024 --replication-factor 3 I was expecting this topic to be created with 1024 total partitions across 3 of my kafka brokers. Instead it seems like it ended up creating 1024 * 3 = 3072 partitions. Is this a bug? I got the number of partitions running the code from the SimpleConsumer example at https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example specifically the findLeader(...) function. I just count the number of partitionsMetadata objects received. That seems to give me the number 3072. The relevant snippet looks like: List<TopicMetadata> metaData = resp.topicsMetadata(); for (TopicMetadata item : metaData) { for (PartitionMetadata part : item.partitionsMetadata()) { numPartitions++; } } Given I am running the above snippet for every broker, should I not expect to see the numPartitions as 1024? I am assuming that item.partitionsMetadata() only returns PartitionMetadata for the partitions this broker is responsible for? If not then how do I exactly figure out what partitions this broker is the leader for? I am using the SimpleConsumer and my logic is getting completely muddled because of this problem. Also is there a way through the kafka console scripts to find out the number of partitions for a given topic? Thanks, Rajiv
