[
https://issues.apache.org/jira/browse/HDDS-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duong updated HDDS-9668:
------------------------
Description:
Today EC with GRPC copies input data buffer several times:
* when parsing protobuf objects from the input network stream, data is copied
from the original netty direct buffer to heap.
* when writing data to files, the heap buffer needs to be copied to direct
buffers by NIO.
The cost of buffer copies themself is inefficient, it comes with GC
consequences of cleaning up the intermediate buffers. Those together slow EC
I/O down.
!ec-ncopies.png|width=807,height=292!
We can use the ZeroCopyMarshaller from RATIS-1932 to avoid all those issues.
was:
Today EC with GRPC copies input data buffer several times:
* when parsing protobuf objects from the input network stream, data is copied
from the original netty direct buffer to heap.
* when writing data to files, the heap buffer needs to be copied to direct
buffers by NIO.
The costs of buffer copies themself is inefficient, and the GC consequences of
cleaning up the intermediate buffers slow the EC I/O down.
!ec-ncopies.png!
> Zero-Copy in EC GRPC
> --------------------
>
> Key: HDDS-9668
> URL: https://issues.apache.org/jira/browse/HDDS-9668
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Duong
> Priority: Major
> Attachments: datanode-on-write-ec-ncopies.html, ec-ncopies.png
>
>
> Today EC with GRPC copies input data buffer several times:
> * when parsing protobuf objects from the input network stream, data is
> copied from the original netty direct buffer to heap.
> * when writing data to files, the heap buffer needs to be copied to direct
> buffers by NIO.
> The cost of buffer copies themself is inefficient, it comes with GC
> consequences of cleaning up the intermediate buffers. Those together slow EC
> I/O down.
> !ec-ncopies.png|width=807,height=292!
> We can use the ZeroCopyMarshaller from RATIS-1932 to avoid all those issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
