[
https://issues.apache.org/jira/browse/HDFS-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289262#comment-14289262
]
Li Bo commented on HDFS-7653:
-----------------------------
Hi, Zhe , thanks for your comments.
1. BlockReader contains some client related methods and maybe we will add
some new methods to the interface so I choose to create a new one.
2. We will create a new dfs inputstream named DFSStripeInputStream and it
uses BlockGroupReader to read data from different datanodes. BlockGroupReader
contains a set of BlockReaders. I will upload code to HDFS-7545 in several
days. Current implementation tries to make client and datanode EC
encoding/decoding work in the same model.
3. Agree that we should simplify the logic of BlockWriter between
datanodes. I will optimize the code later. I am not very clear why
FSOutputSummer is not appropriate for a block writer, it contains a data buffer
and checksum buffer.
4. We can have a further discussion after code is uploaded to HDFS-7545. I
will refer to RSStriper logic in QFS to optimize current implementation.
Nits:
1. we can read a byte and put to buf immediately. The position of buf need
to remain the same, I will fix it and add unit test.
2. I will change the unit test class name later. I generated the patch
after all test cases pass. getBlockFile() is a static method and other classes
also use it in the way of MiniDFSCluster.getBlockFile. You can have a further
check.
> Block Readers and Writers used in both client side and datanode side
> --------------------------------------------------------------------
>
> Key: HDFS-7653
> URL: https://issues.apache.org/jira/browse/HDFS-7653
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Li Bo
> Assignee: Li Bo
> Attachments: BlockReadersWriters.patch
>
>
> There're a lot of block read/write operations in HDFS-EC, for example, when
> client writes a file in striping layout, client has to write several blocks
> to several different datanodes; if a datanode wants to do an
> encoding/decoding task, it has to read several blocks from itself and other
> datanodes, and writes one or more blocks to itself or other datanodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
