[
https://issues.apache.org/jira/browse/HDFS-8147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14501488#comment-14501488
]
surendra singh lilhore commented on HDFS-8147:
----------------------------------------------
Thanks [~szetszwo] for comments
{quote}
I think the problem is that when the Mover schedule the moves DN1 ARCHIVE does
not has the block. When dispatching the blocks, one of them is dispatched
successfully but the other will fail.
{quote}
Based my understanding Mover will move only one replica for block at a time in
one iteration. It will not move all replicas for same block concurrently,
that’s why no chance of conflict in dispatching.
{quote}
We already have block.isLocatedOn(target) in isGoodBlockCandidate(..). So I
think the patch may not help. Could you test it?
{quote}
I already test this patch and its working for me.
Yes we have {{block.isLocatedOn(target)}} in {{isGoodBlockCandidate(..)}}, but
Mover will not use this API.
After selecting target block, mover will call
{{dispatcher.executePendingMove(..)}} and then {{executePendingMove(..)}}
directly call {{p.dispatch()}}.
{code}
public void executePendingMove(final PendingMove p) {
// move the block
moveExecutor.execute(new Runnable() {
@Override
public void run() {
p.dispatch();
}
});
}
{code}
> Mover should not select the DN storage as target where already same replica
> exists.
> -----------------------------------------------------------------------------------
>
> Key: HDFS-8147
> URL: https://issues.apache.org/jira/browse/HDFS-8147
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover
> Affects Versions: 2.6.0
> Reporter: surendra singh lilhore
> Assignee: surendra singh lilhore
> Attachments: HDFS-8147.patch
>
>
> *Scenario:*
> 1. Three DN cluster. For DNs storage type is like this.
> DN1 : DISK,ARCHIVE
> DN2 : DISK
> DN3 : DISK,ARCHIVE (All DNs are in same rack)
> 2. One file with two replicas (In DN1 and DN2)
> 3. Set file storage policy COLD
> 4. Now execute Mover.
> *Expected Result:* File blocks should move in DN1:ARCHIVE and DN3:ARCHIVE
> *Actual Result:* {{chooseTargetInSameNode()}} move D1:DISK block to
> D1:ARCHIVE, but in next iteration {{chooseTarget()}} for same rake is
> selecting again DN1:ARCHIVE for target where already same block exists.
> {{chooseTargetInSameNode()}} and {{chooseTarget()}} should not select the
> node as target where already same replica exists.
> *Logs*
> {code}
> 15/04/15 10:47:17 WARN balancer.Dispatcher: Failed to move
> blk_1073741852_1028 with size=11990 from 10.19.92.74:50010:DISK to
> 10.19.92.73:50010:ARCHIVE through 10.19.92.73:50010: Got error, status
> message opReplaceBlock
> BP-1258709199-10.19.92.74-1428292615636:blk_1073741852_1028 received
> exception
> org.apache.hadoop.hdfs.server.datanode.ReplicaAlreadyExistsException: Replica
> FinalizedReplica, blk_1073741852_1028, FINALIZED
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
