[
https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318358#comment-14318358
]
Kai Zheng commented on HDFS-7729:
---------------------------------
I just took a quick look at the new patch checking how it uses the raw coder.
Some comments:
# Similar to {{blockGroupDataBlocks}}, we can have {{blockGroupParityBlocks}},
instead of {{blockGroupSize}}.
# Better to call this version, encode(ECChunk[], ECChunk[]), which can be more
efficient. Sure we can get this done when change to use block erasure coder as
follow up.
# In {{void encode(ByteBuffer[] buffers)}}, looks like data buffers and parity
buffers are combined together so we need to split. Why we combine them ? For a
certain file, the stripping EC schema is fixed, we can have fixed data and
parity buffers array as members for each time to fill, to prepare for the
encoding.
# We need to call {{release()}} for the coder to release underlying encoding
buffers or resources.
> Add logic to DFSOutputStream to support writing a file in striping layout
> --------------------------------------------------------------------------
>
> Key: HDFS-7729
> URL: https://issues.apache.org/jira/browse/HDFS-7729
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Li Bo
> Assignee: Li Bo
> Attachments: Codec-tmp.patch, HDFS-7729-001.patch,
> HDFS-7729-002.patch, HDFS-7729-003.patch, HDFS-7729-004.patch,
> HDFS-7729-005.patch, HDFS-7729-006.patch, HDFS-7729-007.patch,
> HDFS-7729-008.patch
>
>
> If client wants to directly write a file striping layout, we need to add some
> logic to DFSOutputStream. DFSOutputStream needs multiple DataStreamers to
> write each cell of a stripe to a remote datanode.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
