[
https://issues.apache.org/jira/browse/HADOOP-14907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181044#comment-16181044
]
Daryn Sharp commented on HADOOP-14907:
--------------------------------------
You have a leak, but the screenshot isn't showing the leak. The highlighted
strings are both in the same UGI instance.
Screenshot-1 shows a {{HashMap.Node}} from the {{FileSystem.Cache}}. The
{{HashMap.Node.key}} field above it is a {{FileSystem.Cache.Key}} which
references a {{UserGroupInformation}}. The {{HashMap.Node.value}} field shown
is a {{DistributedFileSystem}} instance, which references the same
{{FileSystem.Cache.Key}}. You can see from the hashcodes that the ugi is
identical.
The problem is you have ~20k {{Subject}} instances. Are you repeatedly
invoking {{UserGroupInformation.createRemoteUser}}?
> Memory leak in FileSystem cache
> -------------------------------
>
> Key: HADOOP-14907
> URL: https://issues.apache.org/jira/browse/HADOOP-14907
> Project: Hadoop Common
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 2.7.4
> Reporter: cen yuhai
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> There is a memory leak in FileSystem cache. It will take a lot of memory.I
> think the root cause is that the equals function in class Key is not right.
> You can see in the screenshot-1.png, the same user etl is in different key...
> And also FileSystem cache should be a LRU cache
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
