[
https://issues.apache.org/jira/browse/HDFS-14531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16858706#comment-16858706
]
Kihwal Lee commented on HDFS-14531:
-----------------------------------
One workaround is to disable DirectoryScanner by setting
"fs.datanode.directoryscan.interval" to -1.
> Datanode's ScanInfo requires excessive memory
> ---------------------------------------------
>
> Key: HDFS-14531
> URL: https://issues.apache.org/jira/browse/HDFS-14531
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Daryn Sharp
> Priority: Major
> Attachments: Screen Shot 2019-05-31 at 12.25.54 PM.png
>
>
> The DirectoryScanner's ScanInfo map consumes ~4.5X memory as replicas as the
> replica map. For 1.1M replicas: the replica map is ~91M while the scan info
> is ~405M.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
