[
https://issues.apache.org/jira/browse/HDFS-8254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589204#comment-14589204
]
Tsz Wo Nicholas Sze commented on HDFS-8254:
-------------------------------------------
Thanks Zhe for the review.
> ... Can we add some Javadocs for the class and methods, and ideally, a
> design description on the JIRA?
Sure, will add the following javadoc.
{code}
/**
* This class is designed for multiple threads to share a
* {@link MultipleBlockingQueue}. Initially, the queue is empty. The earliest
* thread calling poll populates entries to the queue and the other threads
* will wait for it. Once the entries are populated, all the threads can poll
* their entries.
*
* @param <T> the queue entry type.
*/
{code}
> ... It also looks like we could run into a race condition ...
As Walter explained, there is no race condition.
> DFSStripedOutputStream already has the schema ...
The JIRA is for replacing leading streamer implementation. For the other
issues, let's fix them in separated JIRAs.
> New methods in DataStreamer could use some Javadoc.
Do you mean callUpdateBlockForPipeline and callUpdatePipeline? These internal
APIs are both short and straightforward. I believe javadoc is not needed.
> class Coordinator could be renamed to something like StreamersCoordinator,
> just to be more specific.
The class actually is DFSStripedOutputStream.Coordinator.
> The Javadoc of StripedBlockUtil#checkBlocks should say that it checks the two
> blocks are in the same block group
Will fix it.
> In StripedDataStreamer, it is hard to tolerate datanode failure in the
> leading streamer
> ---------------------------------------------------------------------------------------
>
> Key: HDFS-8254
> URL: https://issues.apache.org/jira/browse/HDFS-8254
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
> Attachments: h8254_20150526.patch, h8254_20150526b.patch
>
>
> StripedDataStreamer javadoc is shown below.
> {code}
> * The StripedDataStreamer class is used by {@link DFSStripedOutputStream}.
> * There are two kinds of StripedDataStreamer, leading streamer and ordinary
> * stream. Leading streamer requests a block group from NameNode, unwraps
> * it to located blocks and transfers each located block to its corresponding
> * ordinary streamer via a blocking queue.
> {code}
> Leading streamer is the streamer with index 0. When the datanode of the
> leading streamer fails, the other steamers cannot continue since no one will
> request a block group from NameNode anymore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
