[
https://issues.apache.org/jira/browse/HDFS-8033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14506006#comment-14506006
]
Zhe Zhang commented on HDFS-8033:
---------------------------------
Continuing with Jing's comments:
bq. In DFSInputStream, the reading state can be captured by blockReader,
currentNode, etc. When reading striped blocks, it may be more clear to define
the state as the group of readers and DataNodes.
Good point about maintaining a list of current DataNodes. This will be helpful
to track bad nodes in the parallel reading case. The patch already maintains
{{blockReaders}}. The entire list of {{blockReader}}s are forwarded to the next
striped block group every time {{blockSeekTo}} is called. I guess in the
parallel stateful reading case we can just read from those {{blockReader}}s in
parallel.
bq. Thus instead of declare blockReader, currentNode, etc as protected and keep
updating them in DFSStripedInputStream, I guess it may be better to define
DFSStripedInputStream's own states (using a list of readers etc.). We may want
to override the whole readWithStrategy method in DFSStripedInputStream as well.
Yes, per the above discussion, we should also override {{readWithStrategy}} to
handle {{currentNodes}}
> Erasure coding: stateful (non-positional) read from files in striped layout
> ---------------------------------------------------------------------------
>
> Key: HDFS-8033
> URL: https://issues.apache.org/jira/browse/HDFS-8033
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFS-8033.000.patch, HDFS-8033.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
