[
https://issues.apache.org/jira/browse/HDDS-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duong updated HDDS-10364:
-------------------------
Description:
Jetty's SSL Engine implementation creates a great amount of garbage for GC.
This is because when SSLEngine encodes/decodes any byte[] of input data, it
*creates an entirely new* byte[] as output and *does not reuse* it.
!HDDS-10364.png|width=717,height=271!
This means for every MB of data flows through S3G, there's 1 MB of garbage for
S3G GC to clean up. The cost of GC is linear to the data throughput.
When I do a comparison of writing data from S3 with ratis-streaming vs S3 with
normal async API, ratis-streaming improves the throughput by ~60%, and together
with S3 throughput, S3 GC cost (and consequently CPU util) also improves
significantly.
!s3-write-ratis-grpc.png|width=651,height=558!
!s3-write-ratis-streaming.png|width=653,height=560!
was:
Jetty's SSL Engine implementation creates a great amount of garbage for GC.
This is because when SSLEngine encodes/decodes any byte[] of input data, it
*creates an entirely new* byte[] as output and *does not reuse* it.
!HDDS-10364.png|width=717,height=271!
This means for every MB of data flows through S3G, there's 1 MB of garbage for
S3G GC to clean up. The cost of GC is linear to the data throughput.
When I do a comparison of writing data from S3 with ratis-streaming vs S3 with
normal async API, ratis-streaming improves the throughput by ~60%, and together
with S3 throughput, S3 GC cost (and consequently CPU util) also improves
significantly.
!s3-write-ratis-grpc.pngwidth=709,height=608!
!s3-write-ratis-streaming.png|width=709,height=608!
> S3G: Jetty SSL encryption creates a great load of heat for GC
> -------------------------------------------------------------
>
> Key: HDDS-10364
> URL: https://issues.apache.org/jira/browse/HDDS-10364
> Project: Apache Ozone
> Issue Type: Improvement
> Components: S3, s3gateway
> Reporter: Duong
> Priority: Major
> Attachments: HDDS-10364.png, s3-write-ratis-grpc.png,
> s3-write-ratis-streaming.png, s3_read_allocation.htm, s3_write_allocation.htm
>
>
> Jetty's SSL Engine implementation creates a great amount of garbage for GC.
> This is because when SSLEngine encodes/decodes any byte[] of input data, it
> *creates an entirely new* byte[] as output and *does not reuse* it.
> !HDDS-10364.png|width=717,height=271!
> This means for every MB of data flows through S3G, there's 1 MB of garbage
> for S3G GC to clean up. The cost of GC is linear to the data throughput.
> When I do a comparison of writing data from S3 with ratis-streaming vs S3
> with normal async API, ratis-streaming improves the throughput by ~60%, and
> together with S3 throughput, S3 GC cost (and consequently CPU util) also
> improves significantly.
> !s3-write-ratis-grpc.png|width=651,height=558!
> !s3-write-ratis-streaming.png|width=653,height=560!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
