[jira] [Commented] (HDFS-16220) [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC
[ https://issues.apache.org/jira/browse/HDFS-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458186#comment-17458186 ] JiangHua Zhu commented on HDFS-16220: - Sorry, I am late. Thank you very much for [~shv]'s comment, I will update it later. > [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC > > > Key: HDFS-16220 > URL: https://issues.apache.org/jira/browse/HDFS-16220 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs, namenode >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Major > Labels: pull-request-available > Attachments: debug1.jpg, debug2.jpg > > Time Spent: 3h 10m > Remaining Estimate: 0h > > In INodeMap, NAMESPACE_KEY_DEPTH and NUM_RANGES_STATIC are a fixed value, we > should make it configurable. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16220) [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC
[ https://issues.apache.org/jira/browse/HDFS-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422417#comment-17422417 ] Konstantin Shvachko commented on HDFS-16220: # I don't think we should make Key depth configurable. We should generalize the long[] into a Key class, then we will be able to configure any Key class and load it using a Key factory. Different Key classes then can have different depths. The problem here is that one should be able to construct the keys while loading INodes from the image, so that they could be placed into the right partitions. # Number of partitions should in the end be configurable. It should depend on the number of cores on your server. Increasing the number of partitions does not necessarily increase the parallelism because at any moment the CPU cannot support more threads than the number of cores. So this change is useful, but not critical. > [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC > > > Key: HDFS-16220 > URL: https://issues.apache.org/jira/browse/HDFS-16220 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs, namenode >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Major > Labels: pull-request-available > Attachments: debug1.jpg, debug2.jpg > > Time Spent: 3h 10m > Remaining Estimate: 0h > > In INodeMap, NAMESPACE_KEY_DEPTH and NUM_RANGES_STATIC are a fixed value, we > should make it configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16220) [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC
[ https://issues.apache.org/jira/browse/HDFS-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415859#comment-17415859 ] JiangHua Zhu commented on HDFS-16220: - Here are some phenomena in the debugging process. When numSpaceKeyDepth=2, numRanges=10 !debug1.jpg! When numSpaceKeyDepth=4, numRanges=10 !debug2.jpg! > [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC > > > Key: HDFS-16220 > URL: https://issues.apache.org/jira/browse/HDFS-16220 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs, namenode >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Major > Labels: pull-request-available > Attachments: debug1.jpg, debug2.jpg > > Time Spent: 2h 20m > Remaining Estimate: 0h > > In INodeMap, NAMESPACE_KEY_DEPTH and NUM_RANGES_STATIC are a fixed value, we > should make it configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16220) [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC
[ https://issues.apache.org/jira/browse/HDFS-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413853#comment-17413853 ] JiangHua Zhu commented on HDFS-16220: - Thanks [~prasad-acit] for the comment. > [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC > > > Key: HDFS-16220 > URL: https://issues.apache.org/jira/browse/HDFS-16220 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs, namenode >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > In INodeMap, NAMESPACE_KEY_DEPTH and NUM_RANGES_STATIC are a fixed value, we > should make it configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16220) [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC
[ https://issues.apache.org/jira/browse/HDFS-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413768#comment-17413768 ] Renukaprasad C commented on HDFS-16220: --- Thanks [~jianghuazhu] for reporting the issue and the patch. Configuration file has issue, which results many test failures. You may correct it, should be able to get rid of these unwanted results. Also, there are some static issues reported please take a look. [~shv] [~xinglin] when you feel free, can you please take a look at the PR? > [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC > > > Key: HDFS-16220 > URL: https://issues.apache.org/jira/browse/HDFS-16220 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs, namenode >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > In INodeMap, NAMESPACE_KEY_DEPTH and NUM_RANGES_STATIC are a fixed value, we > should make it configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16220) [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC
[ https://issues.apache.org/jira/browse/HDFS-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413635#comment-17413635 ] JiangHua Zhu commented on HDFS-16220: - [~prasad-acit] [~shv], I hope to have some communication with you here. I've seen some tests fail in Jenkins and I'm not sure if these anomalies are temporary. Can you help review it? Thank you very much. > [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC > > > Key: HDFS-16220 > URL: https://issues.apache.org/jira/browse/HDFS-16220 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs, namenode >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > In INodeMap, NAMESPACE_KEY_DEPTH and NUM_RANGES_STATIC are a fixed value, we > should make it configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
