[
https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14540932#comment-14540932
]
Zhe Zhang commented on HDFS-7621:
---------------------------------
bq. DBlock == reportedBlock. DBlock represents reportedBlock.
Good work Walter! After reading the patch in detail I understand your point
now. Basically, for DN related tasks, NN should parse the block group before
issuing the task. This is a good point and I agree with you. This is similar to
how we are invalidating striped blocks now.
The changes on {{BlockManager}} look good to me. Only nit is that
{{convertToBlockWithLocations}} does similar things as the updated {{addBlock}}
method. We should either add some Javadoc for {{convertToBlockWithLocations}}
or try to consolidate them into a single method which handles both striped and
contiguous blocks. I'm +1 on committing this separately. This is a good
improvement to NameNode {{getBlocks}} RPC call.
On {{Dispatcher}}:
# {{nonCollocatedBlock}} needs some Javadoc.
# bq. I don't want to extend DBlock to support BlockGroup, It's confusing.
I understand this rationale but it seems we are already extending {{DBlock}}
for striped block groups. {{nonCollocatedBlock}} only makes sense for striped
block groups and it needs to be understood in the context of block striping. I
still think it's easier to just include an {{final int[] blockIndices}} field
in {{BlocksWithLocations}} so we can easily parse the returned arrays from NN.
A related question I have is why {{Balancer}} is using this
{{BlocksWithLocations}} structure while {{Mover}} uses {{LocatedBlocks}}, more
like a client. Any chance we can consolidate them a little more? [~jingzhao]
could you share more insights on this?
> Erasure Coding: update the Balancer/Mover data migration logic
> --------------------------------------------------------------
>
> Key: HDFS-7621
> URL: https://issues.apache.org/jira/browse/HDFS-7621
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Walter Su
> Labels: HDFS-7285
> Attachments: HDFS-7621.001.patch, HDFS-7621.002.patch
>
>
> Currently the Balancer/Mover only considers the distribution of replicas of
> the same block during data migration: the migration cannot decrease the
> number of racks. With EC the Balancer and Mover should also take into account
> the distribution of blocks belonging to the same block group.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
