[
https://issues.apache.org/jira/browse/HDDS-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17416557#comment-17416557
]
Kaijie Chen commented on HDDS-5685:
-----------------------------------
I have re-done the tests, here is a summary.
The original data is in the google doc.
[https://docs.google.com/document/d/14x974xK-vHFqfx7rW8TWjC2ngIbgEdHtkUe4waO_XxA/edit?usp=sharing]
Test A (single 10GB key)
||API||Min||Max||Avg||
|Async|64s|72s|68s|
|Streaming (MappedByteBuffer)|55s|68s|62s|
|Streaming (FileRegion)|55s|64s|61s|
Note: the initial runs are slower, so test #1 ~ #6 are ignored, the data above
is starting from test #7.
The performance is limited by the read speed of HDD (< 200MB/s).
Test B (24x 1GB keys from 1 client)
||API||Min||Max||Avg||
|Async|48s|54s|51s|
|Streaming (MappedByteBuffer)|32s|>630s|33s|
|Streaming (FileRegion)|36s|39s|38s|
Note: the initial runs are slower, so test #1 ~ #3 are ignored, the data above
is starting from test #4.
Very often, MappedByteBuffer streaming API fails and retrys, that's why there
are >630s results.
After some runs, FileRegion streaming API stucks (hangs), even if restart the
client.
Currently, streaming API is 30% faster than the async API.
12 HDDs in total is 2.4GB/s throughput, and the network is 20Gbps(2.5GB/s) in
theory.
We are writing by Client -> Primary -> Secondary 1 / Secondary 2, theoretically
24GB * 2 / 20Gbps = 20s.
Although we are far away from the limit, maybe the pipeline to Client ->
Primary -> Secondary 1 -> Secondary 2 will be even more faster?
> Ozone Streaming Performance test.
> ---------------------------------
>
> Key: HDDS-5685
> URL: https://issues.apache.org/jira/browse/HDDS-5685
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: mingchao zhao
> Assignee: Kaijie Chen
> Priority: Major
> Attachments: image-2021-08-27-11-07-36-526.png, screenshot-1.png,
> screenshot-2.png
>
>
> We can test and discuss the performance of Ozone Streaming under this JIRA.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
