[
https://issues.apache.org/jira/browse/HDFS-6208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-6208:
--------------------------------
Attachment: HDFS-6208.1.patch
Here is a patch with these changes:
# {{FsDatasetCache}}: Close the block file and checksum file in the happy path,
not just after a failure. POSIX specs dictate that it's OK to close the file
descriptor from which an mmap was obtained and then keep using the mmap, so
this is fine.
# {{MappableBlock}}: Close the {{FileChannel}} obtained for the block file and
checksum file. This diff looks big only because of indentation changes to
introduce a try-finally block.
# {{TestCacheDirectives}}: At the end of every test, remove all cache
directives and wait for 0 blocks cached. This is done so that we know the
mmaps are freed, and thus any underlying file locks are released, before
starting another {{MiniDFSCluster}}. I started out trying to hook this logic
into the main {{DataNode}} shutdown flow, but it was going to be very brittle
coordinating with all of background caching and uncaching threads. It's much
more convenient to just rely on the OS cleanup at process shutdown. This is
only a problem for tests, where one process starts and stops many test clusters.
> DataNode caching can leak file descriptors.
> -------------------------------------------
>
> Key: HDFS-6208
> URL: https://issues.apache.org/jira/browse/HDFS-6208
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.4.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HDFS-6208.1.patch
>
>
> In the DataNode, management of mmap'd/mlock'd block files can leak file
> descriptors.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
