[
https://issues.apache.org/jira/browse/HADOOP-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620356#action_12620356
]
Raghu Angadi commented on HADOOP-3909:
--------------------------------------
This is because of DatanodeDescriptor.numBlocks(). This iterates over *all* the
blocks for a node to count them. So the
the call iterates over all the blocks (including the replicas) every time front
page is loaded.
We either need to make block count optional on the front page or keep a count
of blocks with each datanode descriptor.
> Slow namenode webUI (Taking 1 minute to load on 2000 nodes cluster.)
> --------------------------------------------------------------------
>
> Key: HADOOP-3909
> URL: https://issues.apache.org/jira/browse/HADOOP-3909
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.17.1
> Reporter: Koji Noguchi
>
> Showing the namenode webui takes a long time with large cluster
> and also adds high cpu load to the namenode.
> Is this expected?
> WHILE pulling the namenode webUI, top showed
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 20957 hadoop 16 0 20.3g 19g 4732 S 131 63.7 9767:05 java
> AFTER the pull
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 20957 hadoop 16 0 20.3g 19g 4732 S 24 63.7 9768:12 java
> It would be nice if we can improve this especially since we tend to hit the
> webUI when namenode is having trouble.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
