[
https://issues.apache.org/jira/browse/HDFS-8499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635759#comment-14635759
]
Zhe Zhang commented on HDFS-8499:
---------------------------------
Thanks Nicholas for the comments. Addressing the {{BIUCContiguous}} and
{{BIUCStriped}} comparison first since I think it's the most critical:
bq. Let's check BlockInfoUnderConstructionContiguous and
BlockInfoUnderConstructionStriped in the HDFS-7285 branch. They have a
different way to implement setExpectedLocations and addReplicaIfNotPresent.
It's a good idea to examine the two UC classes more closely. IMO most
differences are caused by the fact that {{BIUCContiguous}} stores {{replicas}}
as an ArrayList and {{BIUCStriped}} uses an array. It's an easy refactor to
convert both to use array. We can actually save some memory by doing that.
[~jingzhao] Please let me know if I missed anything between contiguous and
striped UC logics.
As discussed under HDFS-7969, {{initializeBlockRecovery}} algorithm is the same
for contiguous and striped blocks. Basically, the only special handling that UC
striped blocks need is to properly assign internal block IDs. That can be done
by a single line of code in {{setExpectedLocations}} and
{{addReplicaIfNotPresent}} each. It also needs to support a simple
{{getBlockIndices}} interface (< 10 LoC). The rest of UC logic (in ~300 LoC)
are shared.
If we keep {{BIUC}} as an interface, all changes to UC logic need to be
duplicated to 2 classes. A recent example is HDFS-8344.
bq. Indeed, converting BlockInfoUnderConstruction as a feature increase memory
usage so that it may not be desirable.
This is a good point. But I think the overhead is limited because: 1) The total
number of UC blocks is a small fraction of all blocks; 2) If we store
{{replicas}} in array instead of {{ArrayList}} that will "draw it even".
> Refactor BlockInfo class hierarchy with static helper class
> -----------------------------------------------------------
>
> Key: HDFS-8499
> URL: https://issues.apache.org/jira/browse/HDFS-8499
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 2.7.0
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Fix For: 2.8.0
>
> Attachments: HDFS-8499.00.patch, HDFS-8499.01.patch,
> HDFS-8499.02.patch, HDFS-8499.03.patch, HDFS-8499.04.patch,
> HDFS-8499.05.patch, HDFS-8499.06.patch, HDFS-8499.07.patch,
> HDFS-8499.UCFeature.patch, HDFS-bistriped.patch
>
>
> In HDFS-7285 branch, the {{BlockInfoUnderConstruction}} interface provides a
> common abstraction for striped and contiguous UC blocks. This JIRA aims to
> merge it to trunk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
