Interesting. A valid directory name is of the form topic-partitionId. What's under those invalid directories (e.g, r, re, etc)? We recently fixed a topic name issue (https://issues.apache.org/jira/browse/KAFKA-495). Not sure if it's related. Also, are you using a non-java producer?
Thanks, Jun On Mon, Oct 1, 2012 at 4:12 AM, Holger Buss <hbu...@gmx.de> wrote: > Hi, > > we had the problem, that the start of kafka brokes down, because the > written Log directories are empty, have strange names and doesn't exist in > Zookeeper as a topic. > > Here are a ls from the kafka logs folder: > > /var/lib/kafka/logs/ > drwxr-xr-x 2 kafka users 4096 24. Aug 13:22 r > drwxr-xr-x 2 kafka users 4096 24. Aug 00:54 re > drwxr-xr-x 2 kafka users 4096 29. Aug 01:34 rec > drwxr-xr-x 2 kafka users 4096 12. Sep 05:30 reci > drwxr-xr-x 2 kafka users 4096 22. Aug 01:08 recip > drwxr-xr-x 2 kafka users 4096 11. Aug 06:22 recipi > drwxr-xr-x 2 kafka users 4096 23. Aug 05:12 recipie > drwxr-xr-x 2 kafka users 4096 5. Sep 01:08 recipien > drwxr-xr-x 2 kafka users 4096 22. Aug 00:50 recipient > drwxr-xr-x 2 kafka users 4096 27. Aug 01:13 recipientE > drwxr-xr-x 2 kafka users 4096 4. Sep 01:02 recipientEv > drwxr-xr-x 2 kafka users 4096 13. Aug 05:53 recipientEve > drwxr-xr-x 2 kafka users 4096 23. Aug 01:06 recipientEven > drwxr-xr-x 2 kafka users 4096 4. Sep 01:07 recipientEvent > drwxr-xr-x 2 kafka users 4096 20. Sep 10:32 recipientEvents-0 > drwxr-xr-x 2 kafka users 4096 20. Sep 10:38 recipientEvents-1 > drwxr-xr-x 2 kafka users 4096 20. Sep 09:41 recipientEvents-10 > drwxr-xr-x 2 kafka users 4096 20. Sep 09:26 recipientEvents-11 > drwxr-xr-x 2 kafka users 4096 20. Sep 10:57 recipientEvents-12 > drwxr-xr-x 2 kafka users 4096 20. Sep 11:24 recipientEvents-13 > drwxr-xr-x 2 kafka users 4096 20. Sep 09:41 recipientEvents-14 > drwxr-xr-x 2 kafka users 4096 20. Sep 10:51 recipientEvents-15 > drwxr-xr-x 2 kafka users 4096 20. Sep 11:41 recipientEvents-16 > drwxr-xr-x 2 kafka users 4096 20. Sep 10:06 recipientEvents-17 > drwxr-xr-x 2 kafka users 4096 20. Sep 09:26 recipientEvents-18 > drwxr-xr-x 2 kafka users 4096 20. Sep 12:02 recipientEvents-19 > drwxr-xr-x 2 kafka users 4096 20. Sep 10:28 recipientEvents-2 > drwxr-xr-x 2 kafka users 4096 20. Sep 10:45 recipientEvents-3 > drwxr-xr-x 2 kafka users 4096 20. Sep 09:31 recipientEvents-4 > drwxr-xr-x 2 kafka users 4096 20. Sep 09:03 recipientEvents-5 > > > The error Meesage at kafka startup: > > Picked up _JAVA_OPTIONS: -XX:DefaultInitialRAMFraction=128 > -XX:DefaultMaxRAMFraction=16 > [2012-09-20 12:03:32,509] INFO Starting Kafka server... > (kafka.server.KafkaServer) > [2012-09-20 12:03:32,588] INFO Loading log 'systemEvents-1' > (kafka.log.LogManager) > [2012-09-20 12:03:32,646] INFO Loading the last segment > /var/kafka_1_02/logs/systemEvents-1/00000000000000000000.kafka in mutable > mode, recovery false (kafka.log.Log) > [2012-09-20 12:03:32,675] INFO Loading log 'recipientEvents-3' > (kafka.log.LogManager) > [2012-09-20 12:03:32,680] INFO Loading the last segment > /var/kafka_1_02/logs/recipientEvents-3/00000000011691660453.kafka in > mutable mode, recovery false (kafka.log.Log) > [2012-09-20 12:03:32,681] INFO Loading log 'recipi' (kafka.log.LogManager) > [2012-09-20 12:03:32,697] FATAL Fatal error during KafkaServerStable > startup. Prepare to shutdown (kafka.server.KafkaServerStartable) > java.lang.StringIndexOutOfBoundsException: String index out of range: -1 > at java.lang.String.substring(String.java:1937) > at kafka.utils.Utils$.getTopicPartition(Utils.scala:558) > at kafka.log.LogManager$$anonfun$4.apply(LogManager.scala:71) > at kafka.log.LogManager$$anonfun$4.apply(LogManager.scala:65) > at > scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34) > at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34) > at kafka.log.LogManager.<init>(LogManager.scala:65) > at kafka.server.KafkaServer.startup(KafkaServer.scala:58) > at > kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34) > at kafka.Kafka$.main(Kafka.scala:50) > at kafka.Kafka.main(Kafka.scala) > [2012-09-20 12:03:32,700] INFO Shutting down Kafka server > (kafka.server.KafkaServer) > [2012-09-20 12:03:32,700] INFO shutdown scheduler kafka-logcleaner- > (kafka.utils.KafkaScheduler) > [2012-09-20 12:03:32,701] INFO Kafka server shut down completed > (kafka.server.KafkaServer) > > r > re > rec > reci > recip > recipi > recipie > recipien > recipient > recipientE > recipientEv > recipientEve > recipientEven > recipientEvent > > Who create this empty folders? Possible broken Messages? > > Best regards > Holger Buss >
