[
https://issues.apache.org/jira/browse/HADOOP-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj Das updated HADOOP-6843:
--------------------------------
Attachment: fs-weak-ref.4.patch
An approach to the above problem is to make the value in the map (used to
implement the cache) a soft reference. Then when no one is referencing a
particular FileSystem instance, and the memory is under pressure, the GC can
delete the entry (and unless the Configuration object is referenced from
elsewhere, that would go away too).
Patch for Y20S (not for commit).
> Entries in the FileSystem's Cache could be cleared when they are not used
> -------------------------------------------------------------------------
>
> Key: HADOOP-6843
> URL: https://issues.apache.org/jira/browse/HADOOP-6843
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: fs-weak-ref.4.patch
>
>
> In FileSystem, there is a cache maintained for Filesystem instances. The
> entries in the cache are cleared only when explicit FileSystem.close is
> invoked. Applications are not careful on this aspect. Typically, they do
> FileSystem.get(), operate on the FileSystem, and then they just forget about
> it. Every FileSystem instance stores a reference to the Configuration object
> that it was created with. Over a period of time, as the cache grows, this can
> lead to OOM (we have seen this happening in our hadoop 20S clusters at Yahoo).
> This jira aims at addressing the above issue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
