Kafka Topic Creation issues (via Kafka-Manager with
auto.create.topics.enable = false)
Version: 0.9.0.1
We created a topic "web" via Kafka-Manager (our brokers are configured for
autocreate to be false) and then clicked on Generate Partitions and
according to the tool, the topic has been created cleanly with partitions
assigned correctly to brokers.
However, when we look into the individual broker logs:
[2016-09-15 18:46:10,268] ERROR [ReplicaFetcherThread-5-1006], Error for
partition [web,2] to broker
1006:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,391] ERROR [ReplicaFetcherThread-4-1005], Error for
partition [web,1] to broker
1005:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,391] WARN [Replica Manager on Broker 1001]: While
recording the replica LEO, the partition [web,5] hasn't been created.
(kafka.server.ReplicaManager)
[2016-09-15 18:46:10,407] ERROR [ReplicaFetcherThread-4-1004], Error for
partition [web,16] to broker
1004:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,472] ERROR [ReplicaFetcherThread-11-1003], Error for
partition [web,23] to broker
1003:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,520] WARN [Replica Manager on Broker 1001]: While
recording the replica LEO, the partition [web,29] hasn't been created.
(kafka.server.ReplicaManager)
[2016-09-15 18:46:10,667] ERROR [ReplicaFetcherThread-11-1004], Error for
partition [web,8] to broker
1004:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This
server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
[2016-09-15 18:46:10,895] WARN [Replica Manager on Broker 1001]: While
recording the replica LEO, the partition [web,21] hasn't been created.
(kafka.server.ReplicaManager)
[2016-09-15 18:46:10,931] WARN [Replica Manager on Broker 1001]: While
recording the replica LEO, the partition [web,13] hasn't been created.
(kafka.server.ReplicaManager)
Indeed, when we actually inspect if the partitions were actually created,
we see disparity with what's reported by the tool. (Many partitions are
actually not present on the brokers yet - leading me to believe that the
partitioning process is stuck)
| ZK Id | Broker Id | According to Kafka Manager |
Actual Partitions Created |
|---------|----------------------------|--------------------------------------------|------------------------------------------------|
| 1001 | broker-1001.xxxx.xxxx.xxxx | 1,2,5,8,9,13,16,21,23,29,30,31 |
web-1,web-16,web-2,web-23,web-8,web-9 |
| 1002 | broker-1004.xxxx.xxxx.xxxx | 2,3,6,9,10,14,16,17,22,24,30,31 |
web-10,web-16,web-3,web-9 |
| 1003 | broker-1002.xxxx.xxxx.xxxx | 3,4,7,10,11,15,17,18,23,24,25,31 |
web-10,web-11,web-3,web-4 |
| 1004 | broker-1003.xxxx.xxxx.xxxx | 0,4,5,8,11,12,16,18,19,24,25,26 |
web-12,web-18,web-19,web-25,web-26 |
| 1005 | broker-1005.xxxx.xxxx.xxxx | 1,5,6,9,12,13,17,19,20,25,26,27 |
web-12,web-13,web-19,web-26,web-27,web-5 |
| 1006 | broker-1006.xxxx.xxxx.xxxx | 2,6,7,10,13,14,18,20,21,26,27,28 |
web-27 |
| 1007 | broker-1007.xxxx.xxxx.xxxx | 0,3,7,11,14,15,19,21,22,27,28,29 |
web-21,web-22,web-7 |
| 1008 | broker-1008.xxxx.xxxx.xxxx | 0,1,4,8,12,15,20,22,23,28,29,30 |
web-1,web-15,web-22,web-29,web-8 |
Finally, when we try to repartition the topic we get:
"Yikes! Partition reassignment currently in progress for. Aborting
operation" (This seems expected because the initial partitioning has not
completed for 3 days - so clearly it's stuck somewhere).
We have created topics with the Kafka-Manager tool before without issues,
but this has started happening consistently for the past 2-3 topics that we
tried creating through the tool.
Any help on this is greatly appreciated.
--
Regards,
Apurva
