[
https://issues.apache.org/jira/browse/HDDS-9843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duong updated HDDS-9843:
------------------------
Description:
Root of probkem is S3G uses the same buffer size to copy streams (when from
client connection to Ozone stream and vice-versus).
For writes (put endpoint), the buffer size is 8kb and is not efficient when
copying small objects because that unnecessarily create work for GC.
For reads (get endpoint), the default buffer size is 8kb for full read and 4mb
(?!!!) for range read.
For small objects read/writes, a single S3G instance can handle thousands of
requests. Stream buffer need to be dynamically calculated based on object size.
was:
Root of probkem is S3G uses the same buffer size to copy streams (when from
client connection to Ozone stream and vice-versus).
For writes (put endpoint), the buffer size is 8kb and is not efficient when
copying small objects because that unnecessarily create work for GC.
For reads (get endpoint), the default buffer size is 4mb.
For small objects read/writes, a single S3G instance can handle thousands of
requests. Stream buffer need to be dynamically calculated based on object size.
> S3: Ozone S3 gateway memory utilization high on while writing small objects
> ---------------------------------------------------------------------------
>
> Key: HDDS-9843
> URL: https://issues.apache.org/jira/browse/HDDS-9843
> Project: Apache Ozone
> Issue Type: Improvement
> Components: S3, s3gateway
> Reporter: Duong
> Priority: Major
>
> Root of probkem is S3G uses the same buffer size to copy streams (when from
> client connection to Ozone stream and vice-versus).
> For writes (put endpoint), the buffer size is 8kb and is not efficient when
> copying small objects because that unnecessarily create work for GC.
> For reads (get endpoint), the default buffer size is 8kb for full read and
> 4mb (?!!!) for range read.
> For small objects read/writes, a single S3G instance can handle thousands of
> requests. Stream buffer need to be dynamically calculated based on object
> size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
