[
https://issues.apache.org/jira/browse/KAFKA-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196328#comment-14196328
]
Pradeep commented on KAFKA-1738:
--------------------------------
yes.. This is happening in 8.2.0-beta version which we are currently on. I had
two topic in kakfa before killing the service and i added three more topics
(ATopic, BTopic and CTopic) and all of them got created successfully as you can
see below. When i list it i can see the topics in kakfa. But if i see the data
folder in kafka, i dont see the partition folders/files for them. Let me know
if you need more details.
[root@dmipvm temp]# service kafka status
Kafka is running as 19396.
LISTEN on tcp port=9999
LISTEN on tcp port=53536
LISTEN on tcp port=9092
LISTEN on tcp port=48330
[root@dmipvm temp]# kill -9 19396
[root@dmipvm temp]# service kafka start
Starting kafka ... STARTED.
[root@dmipvm temp]# /apps/kafka/bin/kafka-topics.sh --create --topic ATopic
--partitions 1 --replication-factor 1 --zookeeper localhost:2181
Created topic "ATopic".
[root@dmipvm temp]# /apps/kafka/bin/kafka-topics.sh --create --topic BTopic
--partitions 1 --replication-factor 1 --zookeeper localhost:2181
Created topic "BTopic".
[root@dmipvm temp]# /apps/kafka/bin/kafka-topics.sh --create --topic CTopic
--partitions 1 --replication-factor 1 --zookeeper localhost:2181
Created topic "CTopic".
[root@dmipvm temp]#
[root@DMIPVM kafka]# ls -lrt
total 16
drwxr-xr-x 2 root root 4096 Nov 4 11:32 topic_1-0
drwxr-xr-x 2 root root 4096 Nov 4 11:34 topic_2-0
-rw-r--r-- 1 root root 28 Nov 4 11:35 replication-offset-checkpoint
-rw-r--r-- 1 root root 28 Nov 4 11:36 recovery-point-offset-checkpoint
[root@DMIPVM kafka]# /apps/kafka/bin/kafka-topics.sh --list --zookeeper
localhost:2181
ATopic
BTopic
CTopic
topic_1
topic_2
> Partitions for topic not created after restart from forced shutdown
> -------------------------------------------------------------------
>
> Key: KAFKA-1738
> URL: https://issues.apache.org/jira/browse/KAFKA-1738
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8.1.1, 0.8.2
> Environment: Linux, 2GB RAM, 2 Core CPU
> Reporter: Pradeep
>
> We are using Kafka Topic APIs to create the topic. But in some cases, the
> topic gets created but we don't see the partition specific files and when
> producer/consumer tries to get the topic metadata and it fails with
> exception. Same happens if one tries to create using the command line.
> k.p.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata for
> topic tloader1 -> No partition metadata for topic tloader1 due to
> kafka.common.UnknownTopicOrPartitionException}] for topic [tloader1]: class
> kafka.common.UnknownTopicOrPartitionException
> Steps to reproduce -
> 1. Stop kafka using kill -9 <PID of Kafka>
> 2. Start Kafka
> 3. Create Topic with partition and replication factor of 1.
> 4. Check the response “Created topic <topic_name>”
> 5. Run the list command to verify if its created.
> 6. Now check the data directory of kakfa. There would not be any for the
> newly created topic.
> We see issues when we are creating new topics. This happens randomly and we
> dont know the exact reasons. We see the below logs in controller during the
> time of creation of topics which doesnt have the partition files.
> 2014-11-03 13:12:50,625] INFO [Controller 0]: New topic creation callback for
> [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Controller 0]: New partition creation
> callback for [JobJTopic,0] (kafka.controller.KafkaController)
> [2014-11-03 13:12:50,626] INFO [Partition state machine on Controller 0]:
> Invoking state change to NewPartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,653] INFO [Replica state machine on controller 0]:
> Invoking state change to NewReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,654] INFO [Partition state machine on Controller 0]:
> Invoking state change to OnlinePartition for partitions [JobJTopic,0]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Live assigned replicas for partition [JobJTopic,0] are: [List(0)]
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,654] DEBUG [Partition state machine on Controller 0]:
> Initializing leader and isr for partition [JobJTopic,0] to
> (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2)
> (kafka.controller.PartitionStateMachine)
> [2014-11-03 13:12:50,667] INFO [Replica state machine on controller 0]:
> Invoking state change to OnlineReplica for replicas
> [Topic=JobJTopic,Partition=0,Replica=0] (kafka.controller.ReplicaStateMachine)
> [2014-11-03 13:12:50,794] WARN [Controller-0-to-broker-0-send-thread],
> Controller 0 fails to send a request to broker id:0,host:DMIPVM,port:9092
> (kafka.controller.RequestSendThread)
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
> at kafka.utils.Utils$.read(Utils.scala:381)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:108)
> at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:146)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [2014-11-03 13:12:50,965] ERROR [Controller-0-to-broker-0-send-thread],
> Controller 0 epoch 2 failed to send request
> Name:UpdateMetadataRequest;Version:0;Controller:0;ControllerEpoch:2;CorrelationId:43;ClientId:id_0-host_null-port_9092;AliveBrokers:id:0,host:DMIPVM,port:9092;PartitionState:[JobJTopic,0]
> ->
> (LeaderAndIsrInfo:(Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:2),ReplicationFactor:1),AllReplicas:0)
> to broker id:0,host:DMIPVM,port:9092. Reconnecting to broker.
> (kafka.controller.RequestSendThread)
> java.nio.channels.ClosedChannelException
> at kafka.network.BlockingChannel.send(BlockingChannel.scala:97)
> at
> kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:132)
> at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
