[
https://issues.apache.org/jira/browse/HDFS-8030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194425#comment-15194425
]
Uma Maheswara Rao G commented on HDFS-8030:
-------------------------------------------
[~zhz] Me and Rakesh discussed again for some time on the document. Please find
the points to discuss/address for your opinion.
# As this design tries to convert files into EC mode from normal file layout,
Blockgroups needs to be created later when converting. But block groups
generally we allocate continuous blockids, but here how do we make that
continuous blockids when converting?
# Does this create overheads on memory as we need to track blockGroups
separately and if the blockids are not continuous as discussed in #1
# parity creation by reading whole 6blocks which will be 6*256MB into memory.
Do we need to think more on this point I think. We may need to think like
contiguous blockid but Parity generation based on Stripes?
Blk_0, Blk_1…Blk_5 are the contiguous blocks. We read cell from each block and
just treat as stripe and generate 3 parity. Continue until finishes all all
data in blocks. Need to think more on this.
# Do we support mixed zone? contains both striped files and contiguous EC files?
Please others also review the document and get the feedbacks.
> HDFS Erasure Coding Phase II -- EC with contiguous layout
> ---------------------------------------------------------
>
> Key: HDFS-8030
> URL: https://issues.apache.org/jira/browse/HDFS-8030
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: erasure-coding
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFSErasureCodingPhaseII-20151204.pdf
>
>
> Data redundancy form -- replication or erasure coding, should be orthogonal
> to block layout -- contiguous or striped. This JIRA explores the combination
> of {{Erasure Coding}} + {{Contiguous}} block layout.
> As will be detailed in the design document, key benefits include preserving
> block locality, and easy conversion between hot and cold modes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
