[
https://issues.apache.org/jira/browse/HDFS-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642895#comment-13642895
]
Suresh Srinivas commented on HDFS-4489:
---------------------------------------
bq. I think a good performance test is to create a namespace of 150M paths.
Flood the NN with thousands of concurrent file & directory add/deletes per
second throughout the namespace. Hopefully there is existing benchmark with
those properties.
I think we are talking about hashmap entry addition and deletion during adds
and delete of files, other than increased memory. I am not sure I understand
the cache pollution part of performance impact, given namenode core objects run
into GBs in a large setup.
I am currently running some slive tests. But I do not currently have bandwidth
to setup a namenode with 150M paths (that would require more than 64GB of JVM
heap). Do you have some bandwidth to do these tests?
> Use InodeID as as an identifier of a file in HDFS protocols and APIs
> --------------------------------------------------------------------
>
> Key: HDFS-4489
> URL: https://issues.apache.org/jira/browse/HDFS-4489
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Brandon Li
> Assignee: Brandon Li
> Fix For: 2.0.5-beta
>
>
> The benefit of using InodeID to uniquely identify a file can be multiple
> folds. Here are a few of them:
> 1. uniquely identify a file cross rename, related JIRAs include HDFS-4258,
> HDFS-4437.
> 2. modification checks in tools like distcp. Since a file could have been
> replaced or renamed to, the file name and size combination is no t reliable,
> but the combination of file id and size is unique.
> 3. id based protocol support (e.g., NFS)
> 4. to make the pluggable block placement policy use fileid instead of
> filename (HDFS-385).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
