[
https://issues.apache.org/jira/browse/HDFS-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716210#comment-14716210
]
Vinayakumar B commented on HDFS-8780:
-------------------------------------
Following are the possible configurations for include/exclude lists as
mentioned in the javadoc.
# Host must have been in the include hosts list and the include hosts list must
not be empty.
# Host is decommissioned by remaining in the include hosts list and added into
the exclude hosts list. Name node is updated with the new information by
issuing dfsadmin -refreshNodes command.
# Host is removed from both include hosts and exclude hosts lists. Name node
is updated with the new informationby issuing dfsamin -refreshNodes command
In this list one more important and default possible condition is not
considered.
Which is *Include List is not configured ( i.e. *, allow everyone) and only
exclude list is configured, where decommissioned node is present.*
In the condition checks, #2 is missed, where host is present in both lists.
I think following changes will do
{code}
private void removeDecomNodeFromList(final List<DatanodeDescriptor> nodeList)
{
// If the include list is empty, any nodes are welcomed and it does not
// make sense to exclude any nodes from the cluster. Therefore, no remove.
if (!hostFileManager.hasIncludes() && !hostFileManager.hasExcludes()) {
return;
}
for (Iterator<DatanodeDescriptor> it = nodeList.iterator(); it.hasNext();) {
DatanodeDescriptor node = it.next();
if ((hostFileManager.isExcluded(node) || !hostFileManager
.isIncluded(node)) && node.isDecommissioned()) {
// Include list is not empty, an existing datanode does not appear
// in both include or exclude lists and it has been decommissioned.
// Remove it from the node list.
it.remove();
}
}
}{code}
BUT, After the above fix, Since the node decommission done based these checks
already, and if node is included again, node would not be decommissioned state.
*So I think its Okay to remove entire hostmanager checks and just remove
decommissioned nodes directly from list*
[~andrew.wang], Do you think this is fine?
> Fetching live/dead datanode list with arg true for
> removeDecommissionNode,returns list with decom node.
> -------------------------------------------------------------------------------------------------------
>
> Key: HDFS-8780
> URL: https://issues.apache.org/jira/browse/HDFS-8780
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: J.Andreina
> Assignee: J.Andreina
> Priority: Critical
>
> Current implementation:
> ======================
> DatanodeManager#removeDecomNodeFromList() , Decommissioned node will be
> removed from dead/live node list only if below conditions are met
> I . If the Include list is not empty.
> II. If include and exclude list does not have decommissioned node and node
> state is decommissioned.
> {code}
> if (!hostFileManager.hasIncludes()) {
> return;
> }
> if ((!hostFileManager.isIncluded(node)) &&
> (!hostFileManager.isExcluded(node))
> && node.isDecommissioned()) {
> // Include list is not empty, an existing datanode does not appear
> // in both include or exclude lists and it has been decommissioned.
> // Remove it from the node list.
> it.remove();
> }
> {code}
> As mentioned in javadoc a datanode cannot be in "already decommissioned
> datanode state".
> Following the steps mentioned in javadoc datanode state is "dead" and not
> decommissioned.
> *Can we avoid the unnecessary checks and have check for the node is in
> decommissioned state then remove from node list. ?*
> Please provide your feedback.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
