[
https://issues.apache.org/jira/browse/HDFS-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13065560#comment-13065560
]
John George commented on HDFS-2114:
-----------------------------------
> Matt Foley commented on HDFS-2114:
> ----------------------------------
>
> bq. The first use of {isNodeDown} is necessary because {downnode} could be
> null in cases when we are checking for "Recommission".
>
> Yes, but in this case nodes[j].getName().equals(downnode) will return false
> without any problem.
Also, I forgot to mention the check ensures that hasdown is not incremented.
But, I agree with you that the code does not look as good as it should. I have
attached a modified patch with some more changes.
>
> bq. My understanding is that this assert ensures that the current blk has
> only
> ONE replica that is in decommissioned state.
>
> But it also seems to ensure that the only replica that is down is the LAST
> one
> in the array of nodes for the block,
> due to the way this block works:
> {code}
> int firstDecomNodeIndex = -1;
> DatanodeInfo[] nodes = blk.getLocations();
> for (int j = 0; j < nodes.length; j++) { // for each replica of blk
> ...
> if (nodes[j].isDecommissioned()) {
> if (firstDecomNodeIndex == -1) {
> firstDecomNodeIndex = j;
> }
> continue;
> }
> assertEquals("Decom node is not at the end", firstDecomNodeIndex, -1);
> }
> {code}
> Yet I don't see any code (eg in LocatedBlock.getLocations()) where downed
> nodes are moved to the end of that list. Maybe somewhere else?
Yes I think it checks to ensure that the nodes are moved to the end of that
list. The code is in FSNamesystem.java. The last 5 lines in the following code
is what does it.
{code}
/**
* Get block locations within the specified range.
* @see ClientProtocol#getBlockLocations(String, long, long)
*/
LocatedBlocks getBlockLocations(String clientMachine, String src,
long offset, long length) throws AccessControlException,
FileNotFoundException, UnresolvedLinkException, IOException {
LocatedBlocks blocks = getBlockLocations(src, offset, length, true, true);
if (blocks != null) {
//sort the blocks
DatanodeDescriptor client = host2DataNodeMap.getDatanodeByHost(
clientMachine);
for (LocatedBlock b : blocks.getLocatedBlocks()) {
clusterMap.pseudoSortByDistance(client, b.getLocations());
// Move decommissioned datanodes to the bottom
Arrays.sort(b.getLocations(), DFSUtil.DECOM_COMPARATOR);
}
}
return blocks;
}
{code}
>
> re-commission of a decommissioned node does not delete excess replica
> ---------------------------------------------------------------------
>
> Key: HDFS-2114
> URL: https://issues.apache.org/jira/browse/HDFS-2114
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: John George
> Assignee: John George
> Attachments: HDFS-2114-2.patch, HDFS-2114-3.patch, HDFS-2114.patch
>
>
> If a decommissioned node is removed from the decommissioned list, namenode
> does not delete the excess replicas it created while the node was
> decommissioned.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
