[
https://issues.apache.org/jira/browse/KAFKA-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16804224#comment-16804224
]
ASF GitHub Bot commented on KAFKA-6983:
---------------------------------------
kondetibharat commented on pull request #6517: KAFKA-6983: Fix to close file
handlers before renaming files/directories
URL: https://github.com/apache/kafka/pull/6517
Fix to close file handlers before renaming files / directories and open them
back if required
Following are the file renaming scenarios:
- Files are renamed to .deleted so they can be deleted
- .cleaned files are renamed to .swap as part of Log.replaceSegments flow
- .swap files are renamed to original files as part of Log.replaceSegments
flow
Following are the folder renaming scenarios:
- When a topic is marked for deletion, folder is renamed
- As part of replacing current logs with future logs in LogManager
In above scenarios, if file handles are not closed, we get file access
violation exception
Idea is to close the logs and file segments before doing a rename and open
them back up if required.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Error while deleting segments - The process cannot access the file because it
> is being used by another process
> --------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-6983
> URL: https://issues.apache.org/jira/browse/KAFKA-6983
> Project: Kafka
> Issue Type: Bug
> Components: log
> Affects Versions: 1.1.0
> Environment: Windows 10
> Reporter: wade wu
> Priority: Major
>
> ......
> [2018-06-01 17:00:07,566] ERROR Error while deleting segments for test4-1 in
> dir D:\data\Kafka\kafka-logs (kafka.server.LogDirFailureChannel)
> java.nio.file.FileSystemException:
> D:\data\Kafka\kafka-logs\test4-1\00000000000000000000.log ->
> D:\data\Kafka\kafka-logs\test4-1\00000000000000000000.log.deleted: The
> process cannot access the file because it is being used by another process.
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
> at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:387)
> at
> sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
> at java.nio.file.Files.move(Files.java:1395)
> at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:697)
> at org.apache.kafka.common.record.FileRecords.renameTo(FileRecords.java:212)
> at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:415)
> at kafka.log.Log.kafka$log$Log$$asyncDeleteSegment(Log.scala:1601)
> at kafka.log.Log.kafka$log$Log$$deleteSegment(Log.scala:1588)
> at
> kafka.log.Log$$anonfun$deleteSegments$1$$anonfun$apply$mcI$sp$1.apply(Log.scala:1170)
> at
> kafka.log.Log$$anonfun$deleteSegments$1$$anonfun$apply$mcI$sp$1.apply(Log.scala:1170)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at kafka.log.Log$$anonfun$deleteSegments$1.apply$mcI$sp(Log.scala:1170)
> at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:1161)
> at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:1161)
> at kafka.log.Log.maybeHandleIOException(Log.scala:1678)
> at kafka.log.Log.deleteSegments(Log.scala:1161)
> at kafka.log.Log.deleteOldSegments(Log.scala:1156)
> at kafka.log.Log.deleteRetentionMsBreachedSegments(Log.scala:1228)
> at kafka.log.Log.deleteOldSegments(Log.scala:1222)
> at kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:854)
> at kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:852)
> at
> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
> at scala.collection.immutable.List.foreach(List.scala:392)
> at
> scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
> at kafka.log.LogManager.cleanupLogs(LogManager.scala:852)
> at kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:385)
> at
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:62)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: java.nio.file.FileSystemException:
> D:\data\Kafka\kafka-logs\test4-1\00000000000000000000.log ->
> D:\data\Kafka\kafka-logs\test4-1\00000000000000000000.log.deleted: The
> process cannot access the file because it is being used by another process.
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
> at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
> at
> sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
> at java.nio.file.Files.move(Files.java:1395)
> at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:694)
> ... 32 more
>
> ......
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
