[
https://issues.apache.org/jira/browse/HDFS-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282292#comment-14282292
]
jiangyu commented on HDFS-6727:
-------------------------------
Hi [~eddyxu] . There will be some chance to trigger a bug when you add a
"hotswap" disk. For example, if you remove a disk as disk1 from DN1, the block
on disk1 DN1 should be repicated to other machine, there is a chance for DN1 to
accept the block on disk1 to another disk on DN1 as disk2. Then when you add
disk1 from a hotswap state, the Volumemap in datanode will add replica from
disk1 to replace the same block in disk2 . Code as below:
ReplicaInfo oldReplica = volumeMap.add(bpid, newReplica);
if (oldReplica != null) {
FsDatasetImpl.LOG.warn("Two block files with the same block id exist " +
"on disk: " + oldReplica.getBlockFile() + " and " + blockFile );
}
When you trigger the blockreport, the block on disk1 is 0 currently on namenode
side.So processReport will go into processFirstBlockReport method in
BlockManager.java. Follow the code path , when call BlockInfo.addStorage method
and the StorageInfo in namenode is disk2 don't match the storage report by
disk1, so we just update the storageinfo , and set prev and next node to null
in triplet cause we don't call listInsert on DatanodeStroage.addBlock. We miss
the chance to move the blockInfo to head.
It is okay if you don't restart the datanode. If you restart the datanode,
after you insert delimiter, we keep to move the node to the head, when we
encounter the block we miss to set the next and prev node, npe will throw. And
after that , the triplet on BlockInfo is mess. The datanode can't be use anyone
except you restart your namenode.
Hope i make it clearly.
> Refresh data volumes on DataNode based on configuration changes
> ---------------------------------------------------------------
>
> Key: HDFS-6727
> URL: https://issues.apache.org/jira/browse/HDFS-6727
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: 2.5.0, 2.4.1
> Reporter: Lei (Eddy) Xu
> Assignee: Lei (Eddy) Xu
> Labels: datanode
> Fix For: 2.6.0
>
> Attachments: HDFS-6727.000.delta-HDFS-6775.txt, HDFS-6727.001.patch,
> HDFS-6727.002.patch, HDFS-6727.003.patch, HDFS-6727.004.patch,
> HDFS-6727.005.patch, HDFS-6727.006.patch, HDFS-6727.006.patch,
> HDFS-6727.007.patch, HDFS-6727.008.patch, HDFS-6727.combo.patch,
> patchFindBugsOutputhadoop-hdfs.txt
>
>
> HDFS-1362 requires DataNode to reload configuration file during the runtime,
> so that DN can change the data volumes dynamically. This JIRA reuses the
> reconfiguration framework introduced by HADOOP-7001 to enable DN to
> reconfigure at runtime.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
