HuangTao created HDFS-14419:
-------------------------------
Summary: Avoid repeated calls to the listOpenFiles function
Key: HDFS-14419
URL: https://issues.apache.org/jira/browse/HDFS-14419
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode, performance
Affects Versions: 3.3.0
Reporter: HuangTao
Assignee: HuangTao
Fix For: 3.3.0
`hdfs dfsadmin -listOpenFiles -path /any/path` will request all opened files.
In the NameNode side, the function LeaseManager.java#getUnderConstructionFiles
will be called.
When there are only N(<maxListOpenFilesResponses, default is 100) files meet
the conditions, but the leaseManager contains M(>maxListOpenFilesResponses)
files, we will scan all leases. Finally, the hasMore will be set true and the
openFileEntries contains N files, the scenario will cause listOpenFiles() will
be called again.
If M is greater more than N, the two calls to getUnderConstructionFiles will
impact the NameNode performance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
