[ 
https://issues.apache.org/jira/browse/HDFS-13139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Sakhuja updated HDFS-13139:
------------------------------------
    Description: 
Configuring Azure WASB storage as a default HDFS location which means that 
Hadoop HDFS capacity will be 0. I have default replication as 1 but now when I 
am trying to decommission a node, datanode tries to rebalance some 28KB of data 
to another available datanode. However, our HDFS has 0 capacity and therefore, 
decommissioning fails with below given error:
{code:java}
New node(s) could not be removed from the cluster. Reason Trying to move 
'28672' bytes worth of data to nodes with '0' bytes of capacity is not 
allowed{code}
Getting the information on cluster shows that default local HDFS is still used 
for some KB space which is getting rebalanced whereas available capacity is 0:
{code:java}
"CapacityRemaining" : 0,
 "CapacityTotal" : 0,
 "CapacityUsed" : 131072,
 "DeadNodes" : "{}",
 "DecomNodes" : "{}",
 "HeapMemoryMax" : 1060372480,
 "HeapMemoryUsed" : 147668152,
 "NonDfsUsedSpace" : 0,
 "NonHeapMemoryMax" : -1,
 "NonHeapMemoryUsed" : 75319744,
 "PercentRemaining" : 0.0,
 "PercentUsed" : 100.0,
 "Safemode" : "",
 "StartTime" : 1518241019502,
 "TotalFiles" : 1,
 "UpgradeFinalized" : true,{code}

  was:
Created a Hadoop cluster and configured Azure WASB storage as a default HDFS 
location which means that Hadoop HDFS capacity will be 0. I have default 
replication as 1 but now when I am trying to decommission a node, datanode 
tries to rebalance some 28KB of data to another available datanode. However, 
our HDFS has 0 capacity and therefore, decommissioning fails with below given 
error:
{code:java}
New node(s) could not be removed from the cluster. Reason Trying to move 
'28672' bytes worth of data to nodes with '0' bytes of capacity is not 
allowed{code}
Getting the information on cluster shows that default local HDFS is still used 
for some KB space which is getting rebalanced whereas available capacity is 0:
{code:java}
"CapacityRemaining" : 0,
 "CapacityTotal" : 0,
 "CapacityUsed" : 131072,
 "DeadNodes" : "{}",
 "DecomNodes" : "{}",
 "HeapMemoryMax" : 1060372480,
 "HeapMemoryUsed" : 147668152,
 "NonDfsUsedSpace" : 0,
 "NonHeapMemoryMax" : -1,
 "NonHeapMemoryUsed" : 75319744,
 "PercentRemaining" : 0.0,
 "PercentUsed" : 100.0,
 "Safemode" : "",
 "StartTime" : 1518241019502,
 "TotalFiles" : 1,
 "UpgradeFinalized" : true,{code}


> Default HDFS as Azure WASB tries rebalancing datanode data to HDFS (0% 
> capacity) and fails
> ------------------------------------------------------------------------------------------
>
>                 Key: HDFS-13139
>                 URL: https://issues.apache.org/jira/browse/HDFS-13139
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode, fs/azure, hdfs
>    Affects Versions: 2.7.3
>            Reporter: Abhishek Sakhuja
>            Priority: Major
>
> Configuring Azure WASB storage as a default HDFS location which means that 
> Hadoop HDFS capacity will be 0. I have default replication as 1 but now when 
> I am trying to decommission a node, datanode tries to rebalance some 28KB of 
> data to another available datanode. However, our HDFS has 0 capacity and 
> therefore, decommissioning fails with below given error:
> {code:java}
> New node(s) could not be removed from the cluster. Reason Trying to move 
> '28672' bytes worth of data to nodes with '0' bytes of capacity is not 
> allowed{code}
> Getting the information on cluster shows that default local HDFS is still 
> used for some KB space which is getting rebalanced whereas available capacity 
> is 0:
> {code:java}
> "CapacityRemaining" : 0,
>  "CapacityTotal" : 0,
>  "CapacityUsed" : 131072,
>  "DeadNodes" : "{}",
>  "DecomNodes" : "{}",
>  "HeapMemoryMax" : 1060372480,
>  "HeapMemoryUsed" : 147668152,
>  "NonDfsUsedSpace" : 0,
>  "NonHeapMemoryMax" : -1,
>  "NonHeapMemoryUsed" : 75319744,
>  "PercentRemaining" : 0.0,
>  "PercentUsed" : 100.0,
>  "Safemode" : "",
>  "StartTime" : 1518241019502,
>  "TotalFiles" : 1,
>  "UpgradeFinalized" : true,{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to