Hi Dave,
I'm not entirely certain, but briefly looking at code:
http://search-hadoop.com/c/Hadoop:/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java%7C%7C%252Bebs+%252BhdfsLine
2679
if ((curReplicas == 0) && (num.decommissionedReplicas() > 0)) {
decommissionOnlyReplicas++;
}
If the block replica is available ONLY on the node which is being
decommissioned, then it is labeled as "Blocks with No Live Replicas". I
would think that this number should go down after the NN replicates these
blocks. Is it?
HTH
Ravi.
On Fri, May 4, 2012 at 5:30 AM, David Ginzburg <ginz...@hotmail.com> wrote:
> Thank you,
>
> I am a bit confused about the meaning of the state -"Blocks With No Live
> Replicas".
> Could not find documentation.
> Can you clarify?
>
>
>
> ------------------------------
> Date: Thu, 3 May 2012 10:44:07 -0500
> Subject: Re: Rack decommission
> From: ravihad...@gmail.com
> To: hdfs-user@hadoop.apache.org
>
>
> Hi David,
>
> > If a DN is decommissioned and returns, Does the NM update it's block
> meta data. If it does, How does it decide when to update the meta data map?
>
> I presume you meant "the NN update it's block meta data". The answer is
> yes. Soon as the NN decides that a node is dead / decommissioned and that
> blocks have / will fallen below their replication factor, it will order
> more replications. This is what is going to happen when you decommission
> the nodes.
>
> When the DNs come back, and they report their blocks to the NN, the NN
> will realise that the blocks are over-replicated and will order deletions.
>
> > When I view the "dfsnodelist.jsp?whatNodes=
> DECOMMISSIONING" page, there are many slaves with "Blocks With No Live
> Replicas"
>
> Obviously depends on how loaded your HDFS was.
>
>
>
>
>
> On Thu, May 3, 2012 at 10:04 AM, David Ginzburg <ginz...@hotmail.com>wrote:
>
> Hi,
> I have began decommissioning the servers in the rack.
> When I view the "dfsnodelist.jsp?whatNodes=DECOMMISSIONING" page, there
> are many slaves with "Blocks With No Live Replicas"
> Is it possible , even if I have rack awareness configured ?
>
>
>
> ------------------------------
> From: ginz...@hotmail.com
> To: hdfs-user@hadoop.apache.org; common-u...@hadoop.apache.org
> Subject: Rack decommission
> Date: Wed, 2 May 2012 11:49:41 +0000
>
>
> Hi,
>
> I have a cluster with 4 racks.
> I am scheduled to replace one of them, with slave servers included.
> The cluster is configured with rack awareness and replication factor is 3.
> It is my understanding that on rack aware clusters, block placement policy
> ensures blocks are at least on two racks.
> I am concerned about decommission of whole rack.
>
> I considered decommission each node in the rack separately and let it
> balance automatically , but I am afraid it will take a lot of time.
>
> If a DN is decommissioned and returns, Does the NM update it's block meta
> data. If it does, How does it decide when to update the meta data map?
>
> Is there a suggested way to decommission a whole rack , I may have missed?
>
>
>
>
>
>
>
>
>
