[
https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621577#comment-14621577
]
Tsz Wo Nicholas Sze commented on HDFS-8287:
-------------------------------------------
When a stripping cell is full, the parity packets is computed by the user
client thread so that the user client cannot continue to write data. It has to
wait until all parity packets are generated and enqueued.
Instead of generating parity packets using the user client thread, one (the
fastest streamer) of the parity streamers could generate the parity packets.
Then the user client can continue to write. Of course, we need more buffer to
store one or more old stripping cells and the buffer cannot be released until
the parity packets are generated.
> DFSStripedOutputStream.writeChunk should not wait for writing parity
> ---------------------------------------------------------------------
>
> Key: HDFS-8287
> URL: https://issues.apache.org/jira/browse/HDFS-8287
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Kai Sasaki
>
> When a stripping cell is full, writeChunk computes and generates parity
> packets. It sequentially calls waitAndQueuePacket so that user client cannot
> continue to write data until it finishes.
> We should allow user client to continue writing instead but not blocking it
> when writing parity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
