[
https://issues.apache.org/jira/browse/HDFS-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen O'Donnell resolved HDFS-15621.
--------------------------------------
Resolution: Fixed
> Datanode DirectoryScanner uses excessive memory
> -----------------------------------------------
>
> Key: HDFS-15621
> URL: https://issues.apache.org/jira/browse/HDFS-15621
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 3.4.0
> Reporter: Stephen O'Donnell
> Assignee: Stephen O'Donnell
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0, 3.1.5, 3.2.3
>
> Attachments: Screenshot 2020-10-09 at 14.11.36.png, Screenshot
> 2020-10-09 at 15.20.56.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> We generally work a rule of 1GB heap on a datanode per 1M blocks. For nodes
> with a lot of blocks, this can mean a lot of heap.
> We recently captured a heapdump of a DN with about 22M blocks and found only
> about 1.5GB was occupied by the ReplicaMap. Another 9GB of the heap is taken
> by the DirectoryScanner ScanInfo objects. Most of this memory was alloated to
> strings.
> Checking the strings in question, we can see two strings per scanInfo,
> looking like:
> {code}
> /current/BP-671271071-10.163.205.13-1552020401842/current/finalized/subdir28/subdir17/blk_1180438785
> _106716708.meta
> {code}
> I will update a screen shot from MAT showing this.
> For the first string especially, the part
> "/current/BP-671271071-10.163.205.13-1552020401842/current/finalized/" will
> be the same for every block in the block pool as the scanner is only
> concerned about finalized blocks.
> We can probably also store just the subdir indexes "28" and "27" rather than
> "subdir28/subdir17" and then construct the path when it is requested via the
> getter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
