[
https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14624344#comment-14624344
]
Walter Su commented on HDFS-8287:
---------------------------------
When writes stripe_0: streamer for datablocks #0~#5 start at the same time.
Assume #3 is the fastest. We let #3 do encode(). But encode() one time is very
long.
...
When writes stripe_80: streamer #0~#2, #4, #5 is ready. #3 is non-responding
and becomes the slowest one. The dataQueue of #3 exceeds writeMaxPackets and
blocks client thread.
I hope HADOOP-11540 can successfully reduces encode() time to an acceptable
level. By then, I still prefer using an independent thread to do the encode().
> DFSStripedOutputStream.writeChunk should not wait for writing parity
> ---------------------------------------------------------------------
>
> Key: HDFS-8287
> URL: https://issues.apache.org/jira/browse/HDFS-8287
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Kai Sasaki
>
> When a stripping cell is full, writeChunk computes and generates parity
> packets. It sequentially calls waitAndQueuePacket so that user client cannot
> continue to write data until it finishes.
> We should allow user client to continue writing instead but not blocking it
> when writing parity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
