[
https://issues.apache.org/jira/browse/HDFS-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDFS-14419:
-----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks [~daryn] for the comments and [~marvelrock] for the patch contribution!
> Avoid repeated calls to the listOpenFiles function
> --------------------------------------------------
>
> Key: HDFS-14419
> URL: https://issues.apache.org/jira/browse/HDFS-14419
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode, performance
> Affects Versions: 3.3.0
> Reporter: HuangTao
> Assignee: HuangTao
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HDFS-14419.001.patch, HDFS-14419.002.patch
>
>
> `hdfs dfsadmin -listOpenFiles -path /any/path` will request all opened files.
> In the NameNode side, the function
> LeaseManager.java#getUnderConstructionFiles will be called.
> When there are only N(<maxListOpenFilesResponses, default is 100) files meet
> the conditions, but the leaseManager contains M(>maxListOpenFilesResponses)
> files, we will scan all leases. Finally, the hasMore will be set true and the
> openFileEntries contains N files, the scenario will cause listOpenFiles() be
> called again.
> If M is greater more than N, the two calls to getUnderConstructionFiles will
> impact the NameNode performance.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
