Hi, Is there a clean way to delete kafka logs that haven't expired yet? When I try to just delete the files from the file system, Kafka is still holding onto them so it doesn't free up space. From lsof:
java 4747 kafka 152u REG 8,16 537008063 2252888 /mnt/kafka-logs/dispatch-0/00000000045637816846.kafka (deleted) java 4747 kafka 153u REG 8,16 537039735 2252889 /mnt/kafka-logs/dispatch-0/00000000046174824909.kafka (deleted) java 4747 kafka 154u REG 8,16 536889360 2252890 /mnt/kafka-logs/dispatch-0/00000000046711864644.kafka (deleted) java 4747 kafka 155u REG 8,16 536909503 2252891 /mnt/kafka-logs/dispatch-0/00000000047248754004.kafka (deleted) java 4747 kafka 156u REG 8,16 536879126 2252892 /mnt/kafka-logs/dispatch-0/00000000047785663507.kafka (deleted) java 4747 kafka 157u REG 8,16 536904918 2252893 /mnt/kafka-logs/dispatch-0/00000000048322542633.kafka (deleted) java 4747 kafka 158u REG 8,16 536959487 2252894 /mnt/kafka-logs/dispatch-0/00000000048859447551.kafka (deleted) java 4747 kafka 159u REG 8,16 536890888 2252895 /mnt/kafka-logs/dispatch-0/00000000049396407038.kafka (deleted) java 4747 kafka 160u REG 8,16 536923453 2252896 /mnt/kafka-logs/dispatch-0/00000000049933297926.kafka (deleted) java 4747 kafka 161u REG 8,16 536889637 2252897 /mnt/kafka-logs/dispatch-0/00000000050470221379.kafka (deleted) When I restart the kafka process, it spends a nontrivial amount of time shutting down, doing a large amount of random reads on the disk before dying. I'm storing the logs on an Amazon EBS mount. Is there a way I can delete those files without needing restart? .Carlo
