[
https://issues.apache.org/jira/browse/HDDS-870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704220#comment-16704220
]
Shashikant Banerjee edited comment on HDDS-870 at 11/30/18 3:38 AM:
--------------------------------------------------------------------
Thanks [~jnp] for the review.
{code:java}
It seems error prone to create a bufferList in ChunkGroupOutputStream and share
it in various ChunkOutputStreams within. The two streams may start working on
same buffer?
{code}
Once one ChunkOutpuStream closes, we start writing the next ChunkOutputstream.
There can be no possibility of two underlying streams to act on the same
buffers concurrently.
In case, an exception is encountered which needs to be handled, the data
residing in the buffer has to be moved to the next stream in list containing a
different block. In such cases, data has to be shared among underlying
streams.So, It seems to be making more sense to maintain it in
ChunkGroupOutPutStream rather than each ChunkOutputStream.
The allocation of buffers has been moved to ChunkPutStream so that, the buffers
get allocated only when write is requested, Otherwise, for an empty key, there
will be no allocation of buffers.
was (Author: shashikant):
Thanks [~jnp] for the review.
{code:java}
It seems error prone to create a bufferList in ChunkGroupOutputStream and share
it in various ChunkOutputStreams within. The two streams may start working on
same buffer?
{code}
Once one ChunkOutpuStream closes, we start writing the next ChunkOutputstream.
There can be no possibility of two underlying streams to act on the same
buffers concurrently.
In case, an exception is encountered which needs to be handled, the data
residing in the buffer has to be moved to the next stream in list containing a
different block. In such cases, data has ton be shared among underlying
streams.So, It seems to be making more sense to maintain it in
ChunkGroupOutPutStream rather than each ChunkOutputStream.
The allocation of buffers has been moved to ChunkPutStream so that, the buffers
get allocated only when write is requested, Otherwise, for an empty key, there
will be no allocation of buffers.
> Avoid creating block sized buffer in ChunkGroupOutputStream
> -----------------------------------------------------------
>
> Key: HDDS-870
> URL: https://issues.apache.org/jira/browse/HDDS-870
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Client
> Affects Versions: 0.4.0
> Reporter: Shashikant Banerjee
> Assignee: Shashikant Banerjee
> Priority: Major
> Fix For: 0.4.0
>
> Attachments: HDDS-870.000.patch, HDDS-870.001.patch,
> HDDS-870.002.patch
>
>
> Currently, for a key, we create a block size byteBuffer in order for caching
> data. This can be replaced with an array of buffers of size flush buffer size
> configured for handling 2 node failures as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
