[
https://issues.apache.org/jira/browse/HADOOP-13971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17803031#comment-17803031
]
Shilun Fan commented on HADOOP-13971:
-------------------------------------
Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a
blocker. Retarget 3.5.0.
> Fix memory leak in FileSystem.Cache.Key class
> ---------------------------------------------
>
> Key: HADOOP-13971
> URL: https://issues.apache.org/jira/browse/HADOOP-13971
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, security
> Reporter: Wei-Chiu Chuang
> Priority: Major
>
> Calling {{FileSystem#get(final URI uri, final Configuration conf, final
> String user)}} multiple times can result in memory leak because of the hash
> method implementation of UserGroupInformation. FileSystem always instantiates
> a new FileSystem object despite using the same user name/same URI.
> In the past, other downstream projects work around this bug by either
> disabling cache (set {{fs.%s.impl.disable.cache}} to {{true}}) or call
> {{FileSystem.closeAllForUGI()}} to release resource on demand. (See for
> instance HIVE-3098, YARN-58, TEZ-1585)
> However, neither approach is desirable. The first workaround loses
> performance because it disables cache. This bug was discussed extensively in
> HADOOP-12707, but the proposed workaround {{FileSystem.closeAllForUGI()}} is
> insufficient, because it won't purge the objects from cache due to the same
> hash method implementation bug.
> I would like to file a new jira, knowing that current workarounds do not
> work, and invite more discussion. An ideal approach is to change UGI hash
> method, but it may break many downstream applications, so setting target
> version as 3.0.0-beta
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
