https://issues.apache.org/jira/browse/HADOOP-4103 will provide warnings in near real-time about blocks with zero good replicas (without adding any overhead). We can of course add on to it later to list up to 1000 of under-replicated blocks for e.g.
The current (or at least till 0.20) dfshealth.jsp is pretty bad on NN CPU since iterates over *all* the blocks (without holding the lock). It takes many seconds to load the page if you say 70M blocks. There are many problems with it.. the feature you are asking increases the dependence on that. To do this feature properly we need to essentially do fsck ...
Raghu. Bill Au wrote:
I am in the process of setting up remote monitoring of my Hadoop cluster. I seems to me that the replication status can only be obtained from the command line by the fsck command. Has anyone though about adding replication status to the NameNode web UI in dfshealth.jsp? Or is that something that I really shouldn't worry about since Hadoop will fix things all by itself? Bill
