Hi Yonghui,
Do you still have this happening? If yes, can you tell us a bit more
about your setup? Is there something else that accesses or maybe
deleting these log files? For more context to this question, please read
the discussion related to this here
http://mail-archives.apache.org/mod_mbox/kafka-dev/201501.mbox/%3C54C47E9B.5060401%40gmail.com%3E
-Jaikiran
On Thursday 08 January 2015 11:19 AM, Yonghui Zhao wrote:
CentOS release 6.3 (Final)
2015-01-07 22:18 GMT+08:00 Harsha <ka...@harsha.io>:
Yonghui,
Which OS you are running.
-Harsha
On Wed, Jan 7, 2015, at 01:38 AM, Yonghui Zhao wrote:
Yes and I found the reason rename in deletion is failed.
In rename progress the files is deleted? and then exception blocks
file
closed in kafka.
But I don't know how can rename failure happen,
[2015-01-07 00:10:48,685] ERROR Uncaught exception in scheduled task
'kafka-log-retention' (kafka.utils.KafkaScheduler)
kafka.common.KafkaStorageException: Failed to change the log file
suffix
from to .deleted for log segment 70781650
at
kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:249)
at
kafka.log.Log.kafka$log$Log$$asyncDeleteSegment(Log.scala:636)
at kafka.log.Log.kafka$log$Log$$deleteSegment(Log.scala:627)
at
kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:415)
at
kafka.log.Log$$anonfun$deleteOldSegments$1.apply(Log.scala:415)
at scala.collection.immutable.List.foreach(List.scala:318)
at kafka.log.Log.deleteOldSegments(Log.scala:415)
at
kafka.log.LogManager.kafka$log$LogManager$$cleanupExpiredSegments(LogManager.scala:325)
at
kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:356)
at
kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:354)
at
scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
at
scala.collection.Iterator$class.foreach(Iterator.scala:727)
at
scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at
scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at
scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
at kafka.log.LogManager.cleanupLogs(LogManager.scala:354)
at
kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:141)
at
kafka.utils.KafkaScheduler$$anon$1.run(KafkaScheduler.scala:100)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2015-01-07 13:56 GMT+08:00 Jun Rao <j...@confluent.io>:
Do you mean that the Kafka broker still holds a file handler on a
deleted
file? Do you see those files being deleted in the Kafka log4j log?
Thanks,
Jun
On Tue, Jan 6, 2015 at 4:46 AM, Yonghui Zhao <zhaoyong...@gmail.com>
wrote:
Hi,
We use kafka_2.10-0.8.1.1 in our server. Today we found disk space
alert.
We find many kafka data files are deleted, but still opened by
kafka.
such as:
_yellowpageV2-0/00000000000068170670.log (deleted)
java 8446 root 724u REG 253,2
536937911
26087362
/home/work/data/soft/kafka-0.8/data/_oakbay_v2_search_topic_ypgsearch_yellowpageV2-0/00000000000068818668.log
(deleted)
java 8446 root 725u REG 253,2
536910838
26087364
/home/work/data/soft/kafka-0.8/data/_oakbay_v2_search_topic_ypgsearch_yellowpageV2-0/00000000000069457098.log
(deleted)
java 8446 root 726u REG 253,2
536917902
26087368
/home/work/data/soft/kafka-0.8/data/_oakbay_v2_search_topic_ypgsearch_yellowpageV2-0/00000000000070104914.log
(deleted)
Is there anything wrong or wrong configed?
