[
https://issues.apache.org/jira/browse/HDFS-11896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092037#comment-16092037
]
Konstantin Shvachko edited comment on HDFS-11896 at 7/18/17 7:55 PM:
---------------------------------------------------------------------
Spent some time debugging this. Found two problems:
# When heartbeat expires and {{DatanodeManager}} does {{removeDatanode()}} it
does not reset {{nonDfsUsed}}, see {{DatanodeDescriptor.resetBlocks()}}, and
some other fields, which it should.
This exists on trunk and other versions.
# During registration {{HeartbeatManager.register()}} first calls
{{addDatanode()}}, which increments {{stats}}, then resets DN usage to zeros
via {{updateHeartbeatState()}}.
This was fixed by HDFS-9034, but still remains in branch-2.7.
For branch-2.7 combination of these two bugs makes nonDfsUsed counted twice on
re-registration.
was (Author: shv):
Spent some time debugging this. Found two problems:
# When heartbeat expires and {{DatanodeManager}} does {{removeDatanode()}} it
does not reset {{nonDfsUsed}}, see {{DatanodeDescriptor.resetBlocks()}}, and
some other fields, which it should. This exists on trunk and other versions.
# During registration {{HeartbeatManager.register()}} first calls
{{addDatanode()}}, which increments {{stats}}, then resets DN usage to zeros
via {{updateHeartbeatState()}}. This was fixed by HDFS-9034, but still remains
in branch-2.7.
For branch-2.7 combination of these two bugs makes nonDfsUsed counted twice on
re-registration.
> Non-dfsUsed will be doubled on dead node re-registration in branch-2.7.
> -----------------------------------------------------------------------
>
> Key: HDFS-11896
> URL: https://issues.apache.org/jira/browse/HDFS-11896
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.7.3
> Reporter: Brahma Reddy Battula
> Assignee: Brahma Reddy Battula
> Labels: release-blocker
> Attachments: HDFS-11896-002.patch, HDFS-11896-branch-2.7-001.patch,
> HDFS-11896-branch-2.7-002.patch, HDFS-11896.patch
>
>
> *Scenario:*
> i)Make you sure you've non-dfs data.
> ii) Stop Datanode
> iii) wait it becomes dead
> iv) now restart and check the non-dfs data
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
