Entries in the FileSystem's Cache could be cleared when they are not used
-------------------------------------------------------------------------
Key: HADOOP-6843
URL: https://issues.apache.org/jira/browse/HADOOP-6843
Project: Hadoop Common
Issue Type: Bug
Reporter: Devaraj Das
Fix For: 0.22.0
In FileSystem, there is a cache maintained for Filesystem instances. The
entries in the cache are cleared only when explicit FileSystem.close is
invoked. Applications are not careful on this aspect. Typically, they do
FileSystem.get(), operate on the FileSystem, and then they just forget about
it. Every FileSystem instance stores a reference to the Configuration object
that it was created with. Over a period of time, as the cache grows, this can
lead to OOM (we have seen this happening in our hadoop 20S clusters at Yahoo).
This jira aims at addressing the above issue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
