[
https://issues.apache.org/jira/browse/HDDS-5946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17439249#comment-17439249
]
mingchao zhao commented on HDDS-5946:
-------------------------------------
Hi [~szetszwo] [~shashikant] [~sadanand_shenoy], We may need to re-discusses
the implementation of Retry. Without bufferList, we don't seem to have a good
way to implement Retry. However, it does make the client memory very large.
> The 'Cannot allocate Memory 'problem occurs in client while test write
> -----------------------------------------------------------------------
>
> Key: HDDS-5946
> URL: https://issues.apache.org/jira/browse/HDDS-5946
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: mingchao zhao
> Priority: Major
> Attachments: screenshot-2.png, screenshot-3.png
>
>
> When I tested write performance with the new client, I got the following
> exception
> !screenshot-2.png!
> I found the problem started after we implemented retry in HDDS-5674. In the
> test, I used a single client to write 400 files concurrently. and each file
> 128MB. The client's memory will become much larger than before.
> !screenshot-3.png!
> The reason for this is that bufferList is used in Retry to maintain
> references to all byteBuffers currently being written. The client will hold
> those references to byteBuffer until ack. In the past, all Bytebuffers were
> released after they were sent, but now we need to wait for write successful
> before releasing. So now the client will accumulate more Bytebuffers and need
> more memory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
