Hi Lawrence, Thanks for that! I'll take a look at your patches. Hopefully they can be merged into a future release
Ben -----Original Message----- From: Lawrence Craft [mailto:lawrence.cr...@gmail.com] Sent: 22 March 2018 19:27 To: users@kafka.apache.org Subject: Re: Issue deleting topic on Windows Hi Ben, This sounds very similar to KAFKA-1194, an issue we encountered on our Windows servers at work. Basically, Kafka is unable to rename log files as they're locked by the OS. The bug is here and is still open: https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FKAFKA-1194&data=02%7C01%7CBen.Young%40fisglobal.com%7Cc91c72cdc5374631599708d5902af694%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C636573436557774999&sdata=bKbx6bYMSuIA5mNtPHVkWRwin1W7Uea3R2OzjPkBRIQ%3D&reserved=0. I am not aware of any straightforward workaround. To get around this, we've had to modify the code and run a custom build. You can try the changes from here: https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fkafka%2Fpull%2F4431&data=02%7C01%7CBen.Young%40fisglobal.com%7Cc91c72cdc5374631599708d5902af694%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C636573436557774999&sdata=V0dXffpRMAvBU8vyQH6ZW71WpmuhFGB4uo5vQdMs1cM%3D&reserved=0 Hope this helps! Lawrence On Thu, Mar 22, 2018 at 9:34 AM, Young, Ben <ben.yo...@fisglobal.com> wrote: > Hi, > > Any thoughts on the below? > > Thanks! > > -----Original Message----- > > Sorry, I forgot to mention we're using version: kafka_2.12-1.0.0 > > -----Original Message----- > > Hi, > > We have an issue in our dev environments when running on Windows. To > provide a clean environment for testing we tear down the topics and > recreate them, but deleting the topics is causing the server to crash. > We then need to delete the logs directory for it to be able to > restart, it doesn't seem to be able to recover. > > The backtrace is below. I wonder if something still has a handle open > on the folder before the rename is attempted? > > Thanks, > Ben > > [2018-03-20 09:31:24,969] INFO [ReplicaFetcherManager on broker 0] > Removed fetcher for partitions TradeFeed-3 > (kafka.server.ReplicaFetcherManager) > [2018-03-20 09:31:25,012] ERROR Error while deleting TradeFeed-3 in > dir f:\tmp\kafka-logs. (kafka.server.LogDirFailureChannel) > java.io.IOException: Failed to rename log directory from > f:\tmp\kafka-logs\TradeFeed-3 to f:\tmp\kafka-logs\TradeFeed-3. > 23a377d8b9b042a9853adc5deec53a2f-delete > at kafka.log.LogManager.asyncDelete(LogManager.scala:671) > at kafka.cluster.Partition.$anonfun$delete$1(Partition.scala:178) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:217) > at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:225) > at kafka.cluster.Partition.delete(Partition.scala:173) > at kafka.server.ReplicaManager.stopReplica(ReplicaManager. > scala:341) > at kafka.server.ReplicaManager.$anonfun$stopReplicas$2( > ReplicaManager.scala:373) > at scala.collection.Iterator.foreach(Iterator.scala:929) > at scala.collection.Iterator.foreach$(Iterator.scala:929) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1417) > at scala.collection.IterableLike.foreach(IterableLike.scala:71) > at scala.collection.IterableLike.foreach$(IterableLike.scala:70) > at scala.collection.AbstractIterable.foreach(Iterable.scala:54) > at kafka.server.ReplicaManager.stopReplicas(ReplicaManager. > scala:371) > at kafka.server.KafkaApis.handleStopReplicaRequest( > KafkaApis.scala:190) > at kafka.server.KafkaApis.handle(KafkaApis.scala:104) > at kafka.server.KafkaRequestHandler.run( > KafkaRequestHandler.scala:65) > at java.lang.Thread.run(Unknown Source) > [2018-03-20 09:31:25,013] INFO [ReplicaManager broker=0] Stopping > serving replicas in dir f:\tmp\kafka-logs > (kafka.server.ReplicaManager) > > > The information contained in this message is proprietary and/or > confidential. If you are not the intended recipient, please: (i) > delete the message and all copies; (ii) do not disclose, distribute or > use the message in any manner; and (iii) notify the sender > immediately. In addition, please be aware that any message addressed > to our domain is subject to archiving and review by persons other than the > intended recipient. Thank you. > The information contained in this message is proprietary and/or > confidential. If you are not the intended recipient, please: (i) > delete the message and all copies; (ii) do not disclose, distribute or > use the message in any manner; and (iii) notify the sender > immediately. In addition, please be aware that any message addressed > to our domain is subject to archiving and review by persons other than the > intended recipient. Thank you. > The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
