MaWenJin created HDFS-4309:
------------------------------
Summary: Multithreaded get through the Cache FileSystem Object to
lead LeaseChecker memory leak
Key: HDFS-4309
URL: https://issues.apache.org/jira/browse/HDFS-4309
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.0.2-alpha, 2.0.1-alpha, 0.23.4, 0.23.1, 0.20.205.0
Reporter: MaWenJin
If multiple threads concurrently execute the following methods will result in
the thread fs = createFileSystem (uri, conf) method is called.And create
multiple DFSClient, start at the same time LeaseChecker daemon thread, may not
be able to use shutdownhook close it after the process, resulting in a memory
leak.
private FileSystem getInternal(URI uri, Configuration conf, Key key) throws
IOException{
FileSystem fs = null;
synchronized (this) {
fs = map.get(key);
}
if (fs != null) {
return fs;
}
// this is
fs = createFileSystem(uri, conf);
synchronized (this) { // refetch the lock again
FileSystem oldfs = map.get(key);
if (oldfs != null) { // a file system is created while lock is releasing
fs.close(); // close the new file system
return oldfs; // return the old file system
}
// now insert the new file system into the map
if (map.isEmpty() && !clientFinalizer.isAlive()) {
Runtime.getRuntime().addShutdownHook(clientFinalizer);
}
fs.key = key;
map.put(key, fs);
if (conf.getBoolean("fs.automatic.close", true)) {
toAutoClose.add(key);
}
return fs;
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
