You may also be experiencing HDFS trying to re-replicate the blocks from
the datanode you decommissioned? I'm not sure if your DN decommission
process would have done that already.
I'm not aware of anything that you can pull off the shelf to maintain
locality. Is decommissioning nodes that common of a task for you? That
seems to be a competing goal to preserving locality...
Ted Tuttle wrote:
Hello-
We have had some problems with our cluster after decommissioning nodes. The 2
things that are troubling us are:
1) decommissioned nodes continue to show in master-status page as Dead Region
Servers
2) we are seeing very busy (i.g. CPU and IO) nodes after decommissioning.
Looking at stack traces on the busy RSs indicates the work is compaction.
Our procedure for decommissioning is:
1) shutdown RS, job tracker and then DN
2) remove node for Hadoop slaves file and HBase regionservers file on master
and RSs
So, is there a way to decommission that will preserve locality and also inform
HBase that a node is gone for good?
Thanks,
Ted
