[
https://issues.apache.org/jira/browse/HDFS-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Zhang updated HDFS-8728:
----------------------------
Attachment: Merge-8-inodeFile.patch
Merge-7-replicationMonitor.patch
Merge-6-locatedStripedBlock.patch
Merge-5-blockPlacementPolicies.patch
Merge-4-blockmanagement.patch
Merge-3-blockInfo.patch
Merge-2-ecZones.patch
Merge-1-codec.patch
Attaching sub-patches 1~8 from the consolidated HDFS-7285 patch as background
of this proposed change. Note that they are trunk-based.
The root of this proposed change is the new {{BlockInfo}} hierarchy introduced
by HDFS-8499. Based on that, we are able to simplify striped blocks handling in
several perspectives:
# {{BlockUCContiguous}} and {{BlockInfoUCStriped}} can now share the UC logic.
This can be seen in {{Merge-3-blockInfo.patch}}. That sub-patch also separates
out a new {{StripedBlockStorageOp}} class to handle striping-specific logic.
# By keeping {{BlockInfoUC}} as a subclass of {{BlockInfo}} (instead of a
separate interface as in the branch), we can avoid changing trunk code that
relies on this relationship. This can be seen in
{{Merge-4-blockmanagement.patch}} ("Allocate and manage striped blocks in
NameNode blockmanagement module."). With this change and other pre-merge
efforts (HDFS-8487, HDFS-8608, HDFS-8623), the [rebased consolidated patch |
https://issues.apache.org/jira/secure/attachment/12744101/Consolidated-20150707.patch]
has much fewer intrusive changes to {{blockmanagement}} and {{namenode}}
modules than the current HDFS-7285 consolidated patch:
{code}
Current HDFS-7285:
2532 insertions(+), 1156 deletions(-) in blockmanagement
1826 insertions(+), 444 deletions(-) in namenode
Rebased (including this proposed simplification):
1251 insertions(+), 201 deletions(-) in blockmanagement
1324 insertions(+), 168 deletions(-) in namenode
{code}
# As we [discussed |
https://issues.apache.org/jira/browse/HDFS-7285?focusedCommentId=14600362&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14600362]
under HDFS-7285, this new {{BlockInfo}} hierarchy also requires us to unify
contiguous and striped blocks handling along the direction of HDFS-8058. I
think we should also take this chance to reach a conclusion on this issue. My
current thought is that we can take the HDFS-8058 approach to avoid duplicating
code, and thoroughly address type safety as a follow-on (some thoughts can be
found on HDFS-8655).
The attached sub-patches 1, 2, 5, 6, 7 are mostly the same as the branch; 3, 4,
8 contain the proposed simplification and should be more thoroughly reviewed
([~andrew.wang]: we went over these ideas in offline discussions; could you
take a look at the patches?). Per Jing's suggestion I'm also working on
generating a patch against the current HDFS-7285 branch.
> Erasure coding: revisit and simplify BlockInfoStriped and INodeFile
> -------------------------------------------------------------------
>
> Key: HDFS-8728
> URL: https://issues.apache.org/jira/browse/HDFS-8728
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: Merge-1-codec.patch, Merge-2-ecZones.patch,
> Merge-3-blockInfo.patch, Merge-4-blockmanagement.patch,
> Merge-5-blockPlacementPolicies.patch, Merge-6-locatedStripedBlock.patch,
> Merge-7-replicationMonitor.patch, Merge-8-inodeFile.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
