Xinyl,
Do you have any logs when the kafka-topics.sh unable to create
topic dirs. Apart from this make sure you point to a different
dir other than /tmp/kafka-logs since this dir gets delete when
your machine restarts and not a place to store your topic data.
-Harsha
On Mon, Feb 2, 2015, at 07:03 PM, Xinyi Su wrote:
> Hi,
>
> -bash-4.1$ bin/kafka-topics.sh --zookeeper <zkhosst>:2181 --create
> --topic
> zerg.hydra --partitions 3 --replication-factor 2
> Created topic "zerg.hydra".
>
> -bash-4.1$ ls -lrt /tmp/kafka-logs/zerg.hydra-2
> total 0
> -rw-r--r-- 1 users 0 Feb 3 02:58 00000000000000000000.log
> -rw-r--r-- 1 users 10485760 Feb 3 02:58 00000000000000000000.index
>
> We can see the topic partition directory is created after the shell
> command
> is executed since I have not sent any data yet.
> But this shell command is not always executed successfully, sometimes it
> fails to create the directory for topic-partition.
>
> Besides, the broker number is greater than replication factor in my kafka
> cluster.
>
> Thanks.
> Xinyi
>
> On 2 February 2015 at 22:24, Gwen Shapira <[email protected]> wrote:
>
> > IIRC, the directory is only created after you send data to the topic.
> >
> > Do you get errors when your producer sends data?
> >
> > Another common issue is that you specify replication-factor 3 when you
> > have fewer than 3 brokers.
> >
> > Gwen
> >
> > On Mon, Feb 2, 2015 at 2:34 AM, Xinyi Su <[email protected]> wrote:
> > > Hi,
> > >
> > > I am using Kafka_2.9.2-0.8.2-beta. When I use kafka-topic.sh to create
> > > topic, I observed sometimes the topic is not really created successfully
> > as
> > > the output shows in console.
> > >
> > > Below is my command line:
> > >
> > > # bin/kafka-topics.sh --zookeeper <xxx>:2181 --create --topic zerg.hydra
> > > --partitions 3 --replication-factor 3
> > >
> > > The command prompts "created topic xxx", but local storage directory used
> > > for this topic under "log.dirs" does not created at all. Normally, there
> > > should be some folders like zerg.hydra-0, zerg.hydra-1... just named
> > > according to partion id and assignment policy.
> > >
> > > I come across this issue about four times, the disk is not full and
> > > directory access permission is legal. Do you know about the cause of this
> > > issue?
> > >
> > > Thanks.
> > >
> > > Xinyi
> >
