[
https://issues.apache.org/jira/browse/HDFS-8147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
surendra singh lilhore updated HDFS-8147:
-----------------------------------------
Description:
*Scenario:*
1. Three DN cluster. For DNs storage type is like this.
DN1 : DISK,ARCHIVE
DN2 : DISK
DN3 : DISK,ARCHIVE (All DNs are in same rack)
2. One file with two replicas (In DN1 and DN2)
3. Set file storage policy COLD
4. Now execute Mover.
*Expected Result:* File blocks should move in DN1:ARCHIVE and DN3:ARCHIVE
*Actual Result:* {{chooseTargetInSameNode()}} move D1:DISK block to D1:ARCHIVE,
but in next iteration {{chooseTarget()}} for same rake is selecting again
DN1:ARCHIVE for target where already same block exists.
{{chooseTargetInSameNode()}} and {{chooseTarget()}} should not select the node
as target where already same replica exists.
*Logs*
{code}
15/04/15 10:47:17 WARN balancer.Dispatcher: Failed to move blk_1073741852_1028
with size=11990 from 10.19.92.74:50010:DISK to 10.19.92.73:50010:ARCHIVE
through 10.19.92.73:50010: Got error, status message opReplaceBlock
BP-1258709199-10.19.92.74-1428292615636:blk_1073741852_1028 received exception
org.apache.hadoop.hdfs.server.datanode.ReplicaAlreadyExistsException: Replica
FinalizedReplica, blk_1073741852_1028, FINALIZED
{code}
was:
{code}
15/04/15 10:47:17 WARN balancer.Dispatcher: Failed to move blk_1073741852_1028
with size=11990 from 10.19.92.74:50010:DISK to 10.19.92.73:50010:ARCHIVE
through 10.19.92.73:50010: Got error, status message opReplaceBlock
BP-1258709199-10.19.92.74-1428292615636:blk_1073741852_1028 received exception
org.apache.hadoop.hdfs.server.datanode.ReplicaAlreadyExistsException: Replica
FinalizedReplica, blk_1073741852_1028, FINALIZED
getNumBytes() = 11990
getBytesOnDisk() = 11990
getVisibleLength()= 11990
getVolume() = /home/hadoop/hadoop/hadoop-3.0.0/dfs/data1/current
getBlockFile() =
/home/hadoop/hadoop/hadoop-3.0.0/dfs/data1/current/BP-1258709199-10.19.92.74-1428292615636/current/finalized/subdir0/subdir0/blk_1073741852
unlinked =false already exists on storage ARCHIVE, block move is
failed
{code}
Summary: Mover should not select the DN storage as target where already
same replica exists. (was: Mover : Dispatcher should not set hasFailure for
DDatanode when receive ReplicaAlreadyExistsException)
> Mover should not select the DN storage as target where already same replica
> exists.
> -----------------------------------------------------------------------------------
>
> Key: HDFS-8147
> URL: https://issues.apache.org/jira/browse/HDFS-8147
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
> Affects Versions: 2.6.0
> Reporter: surendra singh lilhore
> Assignee: surendra singh lilhore
>
> *Scenario:*
> 1. Three DN cluster. For DNs storage type is like this.
> DN1 : DISK,ARCHIVE
> DN2 : DISK
> DN3 : DISK,ARCHIVE (All DNs are in same rack)
> 2. One file with two replicas (In DN1 and DN2)
> 3. Set file storage policy COLD
> 4. Now execute Mover.
> *Expected Result:* File blocks should move in DN1:ARCHIVE and DN3:ARCHIVE
> *Actual Result:* {{chooseTargetInSameNode()}} move D1:DISK block to
> D1:ARCHIVE, but in next iteration {{chooseTarget()}} for same rake is
> selecting again DN1:ARCHIVE for target where already same block exists.
> {{chooseTargetInSameNode()}} and {{chooseTarget()}} should not select the
> node as target where already same replica exists.
> *Logs*
> {code}
> 15/04/15 10:47:17 WARN balancer.Dispatcher: Failed to move
> blk_1073741852_1028 with size=11990 from 10.19.92.74:50010:DISK to
> 10.19.92.73:50010:ARCHIVE through 10.19.92.73:50010: Got error, status
> message opReplaceBlock
> BP-1258709199-10.19.92.74-1428292615636:blk_1073741852_1028 received
> exception
> org.apache.hadoop.hdfs.server.datanode.ReplicaAlreadyExistsException: Replica
> FinalizedReplica, blk_1073741852_1028, FINALIZED
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
