[
https://issues.apache.org/jira/browse/HDDS-9536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duong updated HDDS-9536:
------------------------
Description:
Today, datanodes don't use direct buffers for WriteChunk data. When the chunks
are written to disk, NIO converts those buffers to direct ones and the
conversion seems to be very costly (please see attached
[^datanode-on-write2.html]).
Chunk data proto should be serialized from network/ratis using (pooled) direct
buffers. That would avoid lots of extra costs not only from buffer copying but
also from GCing the immediate buffers.
!Screenshot 2023-10-25 at 8.44.16 AM.png|width=853,height=467!
was:
Today, datanodes don't use direct buffers for WriteChunk data. When the chunks
are written to disk, NIO converts those buffers to direct ones and the
conversion seems to be very costly (please see attached
[^datanode-on-write2.html]).
We need chunk data proto to be serialized from network/ratis using (pooled)
direct buffers. That would avoid lots of extra costs not only from buffer copy
but also from GCing the immediate buffers.
!Screenshot 2023-10-25 at 8.44.16 AM.png|width=853,height=467!
> Datanode perf: Copying (heap) buffers is costly
> -----------------------------------------------
>
> Key: HDDS-9536
> URL: https://issues.apache.org/jira/browse/HDDS-9536
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Duong
> Assignee: Tsz-wo Sze
> Priority: Major
> Attachments: Screenshot 2023-10-25 at 8.44.16 AM.png,
> datanode-on-write2.html
>
>
> Today, datanodes don't use direct buffers for WriteChunk data. When the
> chunks are written to disk, NIO converts those buffers to direct ones and the
> conversion seems to be very costly (please see attached
> [^datanode-on-write2.html]).
> Chunk data proto should be serialized from network/ratis using (pooled)
> direct buffers. That would avoid lots of extra costs not only from buffer
> copying but also from GCing the immediate buffers.
> !Screenshot 2023-10-25 at 8.44.16 AM.png|width=853,height=467!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
