[
https://issues.apache.org/jira/browse/HDFS-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530853#comment-13530853
]
Alfred Xu commented on HDFS-4309:
---------------------------------
Agree. I have encountered the same situation.
> Multithreaded get through the Cache FileSystem Object to lead LeaseChecker
> memory leak
> --------------------------------------------------------------------------------------
>
> Key: HDFS-4309
> URL: https://issues.apache.org/jira/browse/HDFS-4309
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 0.20.205.0, 0.23.1, 0.23.4, 2.0.1-alpha, 2.0.2-alpha
> Reporter: MaWenJin
> Labels: patch
> Attachments: HDFS-4309.patch, jmap2.log
>
> Original Estimate: 204h
> Remaining Estimate: 204h
>
> If multiple threads concurrently execute the following methods will result in
> the thread fs = createFileSystem (uri, conf) method is called.And create
> multiple DFSClient, start at the same time LeaseChecker daemon thread, may
> not be able to use shutdownhook close it after the process, resulting in a
> memory leak.
> private FileSystem getInternal(URI uri, Configuration conf, Key key) throws
> IOException{
> FileSystem fs = null;
> synchronized (this) {
> fs = map.get(key);
> }
> if (fs != null) {
> return fs;
> }
> // this is
> fs = createFileSystem(uri, conf);
> synchronized (this) { // refetch the lock again
> FileSystem oldfs = map.get(key);
> if (oldfs != null) { // a file system is created while lock is
> releasing
> fs.close(); // close the new file system
> return oldfs; // return the old file system
> }
> // now insert the new file system into the map
> if (map.isEmpty() && !clientFinalizer.isAlive()) {
> Runtime.getRuntime().addShutdownHook(clientFinalizer);
> }
> fs.key = key;
> map.put(key, fs);
> if (conf.getBoolean("fs.automatic.close", true)) {
> toAutoClose.add(key);
> }
> return fs;
> }
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
