[
https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902373#comment-14902373
]
Kai Sasaki commented on HDFS-8287:
----------------------------------
There seems to be three cases in roughly divided.
* Smaller than one cell size.
* Equal size to cell size. (current 64KB)
* Larger than one cell size.
At first case, {{writeParityCells}} is called only once inside {{closeImpl}}.
Second case, {{writeParityCells}} is called only once other than {{closeImpl}}.
Last case, {{writeParityCells}} is called multiple times. So how can we handle
these cases generally. Current patch (first take and next submit approach) can
only handle the last case.
When we use double buffer, we have to take encoded buffer every other cells. So
it is impossible to collect all at once at {{closeImpl}}. Please could you give
me any other good way or suggestions? Thank you.
> DFSStripedOutputStream.writeChunk should not wait for writing parity
> ---------------------------------------------------------------------
>
> Key: HDFS-8287
> URL: https://issues.apache.org/jira/browse/HDFS-8287
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Kai Sasaki
> Attachments: HDFS-8287-HDFS-7285.00.patch,
> HDFS-8287-HDFS-7285.01.patch, HDFS-8287-HDFS-7285.02.patch,
> HDFS-8287-HDFS-7285.03.patch, HDFS-8287-HDFS-7285.04.patch,
> HDFS-8287-HDFS-7285.05.patch, HDFS-8287-HDFS-7285.06.patch,
> HDFS-8287-HDFS-7285.07.patch, HDFS-8287-HDFS-7285.08.patch,
> HDFS-8287-HDFS-7285.09.patch, HDFS-8287-HDFS-7285.10.patch,
> HDFS-8287-HDFS-7285.11.patch, HDFS-8287-HDFS-7285.WIP.patch,
> HDFS-8287-performance-report.pdf, h8287_20150911.patch, jstack-dump.txt
>
>
> When a stripping cell is full, writeChunk computes and generates parity
> packets. It sequentially calls waitAndQueuePacket so that user client cannot
> continue to write data until it finishes.
> We should allow user client to continue writing instead but not blocking it
> when writing parity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
