[
https://issues.apache.org/jira/browse/HDDS-14117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18044228#comment-18044228
]
Tsz-wo Sze commented on HDDS-14117:
-----------------------------------
Added fileRead, which can be >10 GB/s. It is reading from OS cache.
{code}
---------------------------------------------------------
key5 with 128M (=134217728) bytes and 16k (=16384) bytesPerChecksum
createStreamKey: 445.42 MB/s ( 0.287 s, buffer 32MB (=33554432), keySize
128.00 MB)blockFile:
/Users/szetszwo/ozone/ozone-fork/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-00e1b71f-ad03-4687-a4fc-7f64442b4dfb/ozone-metadata/datanode-1/data-0/hdds/00e1b71f-ad03-4687-a4fc-7f64442b4dfb/current/containerDir0/3/chunks/115816896921600006.block
md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
fileRead: 12572.70 MB/s ( 0.010 s, buffer 32M (=33554432), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
nonStreamRead: 875.97 MB/s ( 0.146 s, buffer 32M (=33554432), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
streamRead: 4230.90 MB/s ( 0.030 s, buffer 32M (=33554432), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
fileRead: 13178.72 MB/s ( 0.010 s, buffer 8M (=8388608), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
nonStreamRead: 861.84 MB/s ( 0.149 s, buffer 8M (=8388608), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
streamRead: 3830.95 MB/s ( 0.033 s, buffer 8M (=8388608), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
fileRead: 13549.45 MB/s ( 0.009 s, buffer 1M (=1048576), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
nonStreamRead: 871.68 MB/s ( 0.147 s, buffer 1M (=1048576), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
streamRead: 2520.86 MB/s ( 0.051 s, buffer 1M (=1048576), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
fileRead: 6360.18 MB/s ( 0.020 s, buffer 4k (=4096), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
nonStreamRead: 115.77 MB/s ( 1.106 s, buffer 4k (=4096), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
streamRead: 1463.89 MB/s ( 0.087 s, buffer 4k (=4096), keySize
128.00 MB) md5=75 28 76 94 01 E4 D2 65 33 DB 34 7A 02 85 F3 4D
{code}
> Add nonStreamRead and fileRead cases to tests.
> ----------------------------------------------
>
> Key: HDDS-14117
> URL: https://issues.apache.org/jira/browse/HDDS-14117
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: test
> Reporter: Tsz-wo Sze
> Assignee: Tsz-wo Sze
> Priority: Major
>
> For comparison, we add nonStreamRead and fileRead cases to TestStreamRead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
