[
https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14543136#comment-14543136
]
Walter Su commented on HDFS-7621:
---------------------------------
Thanks [~zhz] for helpful comments.
>...convertToBlockWithLocations does similar things as the updated addBlock
>method
clean code & add Javadoc. It's more clear. They're different logic.
>... I still think it's easier to just include an final int[] blockIndices
parsing at NN has smaller code. Also the changes to Dispatcher is small.
>...nonCollocatedBlock only makes sense for striped block group
add javadoc to nonCollocatedBlock.
You can think of it as normal block need to avoid collocation. We can implement
collocation the same way in the future if we have requirement.
>...why Balancer is using this BlocksWithLocations structure while Mover uses
>LocatedBlocks
Balancer handles block and doesn't know file. Balancer gets blocks from Node.
Mover knows file. Mover gets blocks from file.
>...moveBlockAcrossStorage requires Mover send correct numOfBytes, otherwise DN
>reject the command.
My concern is {{moveBlockAcrossStorage}}, HDFS-8289 can't solve the problem.
Balancer doesn't know file. Is it necessary to check corruption when moving
across storage inside DN?
> Erasure Coding: update the Balancer/Mover data migration logic
> --------------------------------------------------------------
>
> Key: HDFS-7621
> URL: https://issues.apache.org/jira/browse/HDFS-7621
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Walter Su
> Labels: HDFS-7285
> Attachments: HDFS-7621.001.patch, HDFS-7621.002.patch
>
>
> Currently the Balancer/Mover only considers the distribution of replicas of
> the same block during data migration: the migration cannot decrease the
> number of racks. With EC the Balancer and Mover should also take into account
> the distribution of blocks belonging to the same block group.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
