You can find attached the benchmarks on the host and guest. I find the
differences not so big though...
Le 09/09/2021 à 13:40, Nir Soffer a écrit :
There are few issues with this test:
- you don't use oflag=direct or conv=fsync, so this may test copying data
to the host page cache, instead of writing data to storage
- This tests only sequential write, which is the best case for any
kind of storage
- Using synchronous I/O - every write wait for the previous write
completion
- Using single process
- 2g is too small, may test your cache performance
Try to test using fio - attached fio script that tests sequential and
random io with
various queue depth.
You can use it like this:
fio --filename=/path/to/fio.data --output=test.out bench.fio
Test both on the host, and in the VM. This will give you more detailed
results that may help to evaluate the issue, and it may help Gluster
folks to advise on tuning your storage.
Nir
--
téïcée <https://www.teicee.com/?pk_campaign=Email> *Mathieu Valois*
Bureau Caen: Quartier Kœnig - 153, rue Géraldine MOCK - 14760
Bretteville-sur-Odon
Bureau Vitré: Zone de la baratière - 12, route de Domalain - 35500 Vitré
02 72 34 13 20 | www.teicee.com <https://www.teicee.com/?pk_campaign=Email>
téïcée sur facebook <https://www.facebook.com/teicee> téïcée sur twitter
<https://twitter.com/Teicee_fr> téïcée sur linkedin
<https://www.linkedin.com/company/t-c-e> téïcée sur viadeo
<https://fr.viadeo.com/fr/company/teicee> Datadocké
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T)
1024KiB-1024KiB, ioengine=libaio, iodepth=8
...
seq-read: (g=1): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T)
1024KiB-1024KiB, ioengine=libaio, iodepth=8
...
rand-write-qd32: (g=2): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=32
...
rand-read-qd32: (g=3): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=32
...
rand-write-qd1: (g=4): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=1
rand-read-qd1: (g=5): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=1
fio-3.19
Starting 18 processes
seq-write: Laying out IO file (1 file / 10240MiB)
seq-write: (groupid=0, jobs=4): err= 0: pid=294433: Thu Sep 9 14:30:14 2021
write: IOPS=151, BW=153MiB/s (160MB/s)(4628MiB/30280msec); 0 zone resets
slat (usec): min=43, max=826750, avg=2541.28, stdev=26317.99
clat (msec): min=4, max=1103, avg=207.56, stdev=169.55
lat (msec): min=4, max=1103, avg=210.09, stdev=170.68
clat percentiles (msec):
| 1.00th=[ 22], 5.00th=[ 31], 10.00th=[ 36], 20.00th=[ 53],
| 30.00th=[ 92], 40.00th=[ 133], 50.00th=[ 174], 60.00th=[ 209],
| 70.00th=[ 255], 80.00th=[ 317], 90.00th=[ 447], 95.00th=[ 558],
| 99.00th=[ 818], 99.50th=[ 844], 99.90th=[ 1011], 99.95th=[ 1070],
| 99.99th=[ 1099]
bw ( KiB/s): min= 8172, max=433308, per=100.00%, avg=160585.97,
stdev=20005.36, samples=232
iops : min= 4, max= 420, avg=153.17, stdev=19.58, samples=232
lat (msec) : 10=0.13%, 20=0.72%, 50=18.22%, 100=12.85%, 250=37.87%
lat (msec) : 500=23.85%, 750=5.72%, 1000=1.15%, 2000=0.11%
cpu : usr=0.27%, sys=2.70%, ctx=2384, majf=0, minf=772
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,4600,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
seq-read: (groupid=1, jobs=4): err= 0: pid=294778: Thu Sep 9 14:30:14 2021
read: IOPS=7084, BW=7086MiB/s (7430MB/s)(208GiB/30016msec)
slat (usec): min=6, max=118317, avg=68.62, stdev=596.43
clat (nsec): min=839, max=144267k, avg=4447719.83, stdev=10105727.60
lat (usec): min=57, max=144320, avg=4516.54, stdev=10126.98
clat percentiles (usec):
| 1.00th=[ 178], 5.00th=[ 408], 10.00th=[ 408], 20.00th=[ 416],
| 30.00th=[ 433], 40.00th=[ 445], 50.00th=[ 457], 60.00th=[ 474],
| 70.00th=[ 494], 80.00th=[ 537], 90.00th=[ 29492], 95.00th=[ 31065],
| 99.00th=[ 32113], 99.50th=[ 33162], 99.90th=[ 35914], 99.95th=[ 39060],
| 99.99th=[125305]
bw ( MiB/s): min= 826, max=58638, per=99.77%, avg=7069.26, stdev=3497.32,
samples=236
iops : min= 824, max=58637, avg=7067.41, stdev=3497.32, samples=236
lat (nsec) : 1000=0.14%
lat (usec) : 2=0.08%, 4=0.01%, 20=0.01%, 100=0.46%, 250=0.75%
lat (usec) : 500=71.15%, 750=10.75%, 1000=0.36%
lat (msec) : 2=1.89%, 4=0.02%, 10=0.63%, 20=0.84%, 50=12.90%
lat (msec) : 100=0.01%, 250=0.02%
cpu : usr=0.41%, sys=11.20%, ctx=30465, majf=0, minf=3390
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=212660,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
rand-write-qd32: (groupid=2, jobs=4): err= 0: pid=295141: Thu Sep 9 14:30:14
2021
write: IOPS=228, BW=928KiB/s (951kB/s)(28.1MiB/30971msec); 0 zone resets
slat (usec): min=8, max=800895, avg=1067.23, stdev=24781.44
clat (msec): min=11, max=1565, avg=554.93, stdev=256.34
lat (msec): min=12, max=1565, avg=555.98, stdev=256.45
clat percentiles (msec):
| 1.00th=[ 44], 5.00th=[ 157], 10.00th=[ 220], 20.00th=[ 347],
| 30.00th=[ 422], 40.00th=[ 481], 50.00th=[ 527], 60.00th=[ 592],
| 70.00th=[ 667], 80.00th=[ 776], 90.00th=[ 911], 95.00th=[ 986],
| 99.00th=[ 1234], 99.50th=[ 1267], 99.90th=[ 1284], 99.95th=[ 1502],
| 99.99th=[ 1569]
bw ( KiB/s): min= 28, max= 2944, per=100.00%, avg=932.10, stdev=115.69,
samples=240
iops : min= 4, max= 734, avg=230.08, stdev=28.95, samples=240
lat (msec) : 20=0.14%, 50=1.29%, 100=1.46%, 250=8.69%, 500=34.11%
lat (msec) : 750=33.73%, 1000=18.02%, 2000=4.32%
cpu : usr=0.03%, sys=0.21%, ctx=3729, majf=0, minf=233
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=0,7065,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
rand-read-qd32: (groupid=3, jobs=4): err= 0: pid=296094: Thu Sep 9 14:30:14
2021
read: IOPS=552k, BW=2157MiB/s (2262MB/s)(63.2GiB/30001msec)
slat (nsec): min=1356, max=508918k, avg=5192.02, stdev=265716.52
clat (nsec): min=640, max=1721.8M, avg=229054.00, stdev=4972066.22
lat (usec): min=2, max=1721.9k, avg=234.33, stdev=4979.42
clat percentiles (nsec):
| 1.00th=[ 692], 5.00th=[ 7392], 10.00th=[ 31360],
| 20.00th=[ 92672], 30.00th=[ 128512], 40.00th=[ 148480],
| 50.00th=[ 164864], 60.00th=[ 181248], 70.00th=[ 199680],
| 80.00th=[ 228352], 90.00th=[ 280576], 95.00th=[ 321536],
| 99.00th=[ 634880], 99.50th=[ 1155072], 99.90th=[ 2007040],
| 99.95th=[ 2244608], 99.99th=[173015040]
bw ( MiB/s): min= 829, max= 2529, per=100.00%, avg=2157.05, stdev=111.87,
samples=236
iops : min=212434, max=647558, avg=552203.37, stdev=28639.46,
samples=236
lat (nsec) : 750=2.36%, 1000=0.32%
lat (usec) : 2=0.10%, 4=1.10%, 10=1.73%, 20=2.19%, 50=5.42%
lat (usec) : 100=8.22%, 250=63.64%, 500=13.60%, 750=0.48%, 1000=0.22%
lat (msec) : 2=0.51%, 4=0.07%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2000=0.01%
cpu : usr=17.44%, sys=60.65%, ctx=75348, majf=0, minf=527
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=16564797,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
rand-write-qd1: (groupid=4, jobs=1): err= 0: pid=296386: Thu Sep 9 14:30:14
2021
write: IOPS=55, BW=223KiB/s (229kB/s)(6696KiB/30002msec); 0 zone resets
slat (usec): min=20, max=81257, avg=190.26, stdev=3316.34
clat (usec): min=194, max=105690, avg=17727.89, stdev=13096.14
lat (usec): min=233, max=142461, avg=17918.55, stdev=13680.78
clat percentiles (usec):
| 1.00th=[ 1795], 5.00th=[ 3982], 10.00th=[ 6128], 20.00th=[ 8586],
| 30.00th=[ 10290], 40.00th=[ 12387], 50.00th=[ 14484], 60.00th=[ 16450],
| 70.00th=[ 20055], 80.00th=[ 23987], 90.00th=[ 33162], 95.00th=[ 45351],
| 99.00th=[ 69731], 99.50th=[ 77071], 99.90th=[102237], 99.95th=[105382],
| 99.99th=[105382]
bw ( KiB/s): min= 111, max= 319, per=99.35%, avg=221.54, stdev=46.34,
samples=59
iops : min= 27, max= 79, avg=54.66, stdev=11.60, samples=59
lat (usec) : 250=0.06%, 500=0.42%, 750=0.12%, 1000=0.06%
lat (msec) : 2=0.54%, 4=3.88%, 10=23.06%, 20=42.00%, 50=26.28%
lat (msec) : 100=3.46%, 250=0.12%
cpu : usr=0.09%, sys=0.36%, ctx=1677, majf=0, minf=58
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,1674,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
rand-read-qd1: (groupid=5, jobs=1): err= 0: pid=296633: Thu Sep 9 14:30:14 2021
read: IOPS=39.4k, BW=154MiB/s (161MB/s)(4617MiB/30001msec)
slat (nsec): min=1353, max=99082k, avg=3917.01, stdev=137827.66
clat (nsec): min=640, max=60535k, avg=20869.92, stdev=194315.47
lat (usec): min=2, max=99181, avg=24.88, stdev=239.10
clat percentiles (nsec):
| 1.00th=[ 644], 5.00th=[ 652], 10.00th=[ 652],
| 20.00th=[ 652], 30.00th=[ 660], 40.00th=[ 660],
| 50.00th=[ 684], 60.00th=[ 716], 70.00th=[ 1592],
| 80.00th=[ 1624], 90.00th=[ 105984], 95.00th=[ 118272],
| 99.00th=[ 136192], 99.50th=[ 138240], 99.90th=[ 150528],
| 99.95th=[ 160768], 99.99th=[9633792]
bw ( KiB/s): min=116487, max=172718, per=99.98%, avg=157540.78,
stdev=12396.80, samples=59
iops : min=29121, max=43179, avg=39384.81, stdev=3099.20, samples=59
lat (nsec) : 750=64.74%, 1000=3.13%
lat (usec) : 2=16.07%, 4=0.12%, 10=0.04%, 20=0.15%, 50=0.01%
lat (usec) : 100=4.83%, 250=10.87%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=5.78%, sys=12.61%, ctx=186050, majf=0, minf=61
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1181893,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=153MiB/s (160MB/s), 153MiB/s-153MiB/s (160MB/s-160MB/s), io=4628MiB
(4853MB), run=30280-30280msec
Run status group 1 (all jobs):
READ: bw=7086MiB/s (7430MB/s), 7086MiB/s-7086MiB/s (7430MB/s-7430MB/s),
io=208GiB (223GB), run=30016-30016msec
Run status group 2 (all jobs):
WRITE: bw=928KiB/s (951kB/s), 928KiB/s-928KiB/s (951kB/s-951kB/s), io=28.1MiB
(29.4MB), run=30971-30971msec
Run status group 3 (all jobs):
READ: bw=2157MiB/s (2262MB/s), 2157MiB/s-2157MiB/s (2262MB/s-2262MB/s),
io=63.2GiB (67.8GB), run=30001-30001msec
Run status group 4 (all jobs):
WRITE: bw=223KiB/s (229kB/s), 223KiB/s-223KiB/s (229kB/s-229kB/s), io=6696KiB
(6857kB), run=30002-30002msec
Run status group 5 (all jobs):
READ: bw=154MiB/s (161MB/s), 154MiB/s-154MiB/s (161MB/s-161MB/s), io=4617MiB
(4841MB), run=30001-30001msec
Disk stats (read/write):
dm-21: ios=3261529/26212, merge=0/0, ticks=4094670/7275235,
in_queue=11369905, util=95.25%, aggrios=3262333/26499, aggrmerge=0/0,
aggrticks=4053810/7251327, aggrin_queue=11305137, aggrutil=94.42%
dm-18: ios=3262333/26499, merge=0/0, ticks=4053810/7251327,
in_queue=11305137, util=94.42%, aggrios=1663895/18267, aggrmerge=0/0,
aggrticks=2527558/4306360, aggrin_queue=6833918, aggrutil=94.36%
dm-17: ios=3327737/36463, merge=0/0, ticks=5051948/8609589,
in_queue=13661537, util=94.36%, aggrios=1113206/27777, aggrmerge=0/0,
aggrticks=2259269/2773040, aggrin_queue=5032309, aggrutil=65.79%
dm-15: ios=1286/24248, merge=0/0, ticks=1051/11620, in_queue=12671,
util=2.65%, aggrios=3313657/47664, aggrmerge=61/1480, aggrticks=3137631/53760,
aggrin_queue=3191390, aggrutil=52.34%
dm-4: ios=3313657/47664, merge=61/1480, ticks=3137631/53760,
in_queue=3191390, util=52.34%, aggrios=3313669/47664, aggrmerge=0/0,
aggrticks=3121966/51507, aggrin_queue=3173472, aggrutil=52.24%
sdd: ios=3313669/47664, merge=0/0, ticks=3121966/51507, in_queue=3173472,
util=52.24%
dm-16: ios=25901/34187, merge=0/0, ticks=3632707/8262185,
in_queue=11894892, util=65.79%, aggrios=27671/37083, aggrmerge=0/0,
aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
md125: ios=27671/37083, merge=0/0, ticks=0/0, in_queue=0, util=0.00%,
aggrios=15220/25759, aggrmerge=58800/823861, aggrticks=1658275/1124632,
aggrin_queue=2782907, aggrutil=51.64%
sdf: ios=15090/25795, merge=60713/822931, ticks=1285984/1173500,
in_queue=2459484, util=50.38%
sde: ios=15300/25639, merge=56966/822517, ticks=1638879/926670,
in_queue=2565548, util=48.12%
sdc: ios=15272/25843, merge=58721/826137, ticks=2049963/1273728,
in_queue=3323690, util=51.64%
dm-14: ios=3312432/24896, merge=0/0, ticks=3144050/45315, in_queue=3189365,
util=51.88%
dm-13: ios=53/72, merge=0/0, ticks=3169/3131, in_queue=6300, util=2.39%
seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T)
1024KiB-1024KiB, ioengine=libaio, iodepth=8
...
seq-read: (g=1): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T)
1024KiB-1024KiB, ioengine=libaio, iodepth=8
...
rand-write-qd32: (g=2): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=32
...
rand-read-qd32: (g=3): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=32
...
rand-write-qd1: (g=4): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=1
rand-read-qd1: (g=5): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=1
fio-3.16
Starting 18 processes
seq-write: Laying out IO file (1 file / 10240MiB)
seq-write: (groupid=0, jobs=4): err= 0: pid=46235: Thu Sep 9 14:18:05 2021
write: IOPS=57, BW=58.8MiB/s (61.6MB/s)(1792MiB/30492msec); 0 zone resets
slat (usec): min=53, max=335320, avg=1692.05, stdev=21085.55
clat (msec): min=49, max=2472, avg=557.25, stdev=404.74
lat (msec): min=49, max=2473, avg=558.92, stdev=407.28
clat percentiles (msec):
| 1.00th=[ 125], 5.00th=[ 247], 10.00th=[ 321], 20.00th=[ 338],
| 30.00th=[ 342], 40.00th=[ 342], 50.00th=[ 342], 60.00th=[ 351],
| 70.00th=[ 472], 80.00th=[ 877], 90.00th=[ 1301], 95.00th=[ 1418],
| 99.00th=[ 1636], 99.50th=[ 1838], 99.90th=[ 2467], 99.95th=[ 2467],
| 99.99th=[ 2467]
bw ( KiB/s): min= 7391, max=111568, per=100.00%, avg=71150.39,
stdev=5677.70, samples=183
iops : min= 4, max= 108, avg=68.11, stdev= 5.58, samples=183
lat (msec) : 50=0.06%, 100=0.51%, 250=4.65%, 500=67.40%, 750=6.41%
lat (msec) : 1000=4.37%, 2000=17.91%, >=2000=0.28%
cpu : usr=0.12%, sys=0.13%, ctx=1648, majf=0, minf=60
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.8%, 8=0.2%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,1764,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
seq-read: (groupid=1, jobs=4): err= 0: pid=46315: Thu Sep 9 14:18:05 2021
read: IOPS=5100, BW=5101MiB/s (5349MB/s)(150GiB/30111msec)
slat (usec): min=26, max=4205, avg=86.91, stdev=38.55
clat (nsec): min=815, max=332475k, avg=6186377.89, stdev=26542039.96
lat (usec): min=55, max=332517, avg=6273.58, stdev=26538.34
clat percentiles (nsec):
| 1.00th=[ 980], 5.00th=[ 78336], 10.00th=[ 238592],
| 20.00th=[ 473088], 30.00th=[ 505856], 40.00th=[ 536576],
| 50.00th=[ 577536], 60.00th=[ 634880], 70.00th=[ 692224],
| 80.00th=[ 765952], 90.00th=[ 954368], 95.00th=[ 25559040],
| 99.00th=[154140672], 99.50th=[177209344], 99.90th=[256901120],
| 99.95th=[274726912], 99.99th=[299892736]
bw ( MiB/s): min= 121, max=48079, per=98.89%, avg=5044.70, stdev=3000.15,
samples=240
iops : min= 120, max=48079, avg=5043.07, stdev=3000.18, samples=240
lat (nsec) : 1000=1.33%
lat (usec) : 2=1.38%, 4=0.03%, 10=0.01%, 20=0.02%, 50=0.01%
lat (usec) : 100=2.94%, 250=4.61%, 500=17.75%, 750=49.89%, 1000=12.82%
lat (msec) : 2=2.81%, 4=0.06%, 10=0.16%, 20=1.01%, 50=0.68%
lat (msec) : 100=1.85%, 250=2.54%, 500=0.12%
cpu : usr=0.34%, sys=11.26%, ctx=10087, majf=0, minf=60
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=153582,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
rand-write-qd32: (groupid=2, jobs=4): err= 0: pid=46330: Thu Sep 9 14:18:05
2021
write: IOPS=299, BW=1215KiB/s (1244kB/s)(35.8MiB/30212msec); 0 zone resets
slat (usec): min=5, max=167756, avg=368.13, stdev=6422.10
clat (msec): min=22, max=3049, avg=422.46, stdev=559.64
lat (msec): min=22, max=3049, avg=422.82, stdev=560.40
clat percentiles (msec):
| 1.00th=[ 51], 5.00th=[ 97], 10.00th=[ 103], 20.00th=[ 157],
| 30.00th=[ 209], 40.00th=[ 220], 50.00th=[ 226], 60.00th=[ 234],
| 70.00th=[ 251], 80.00th=[ 359], 90.00th=[ 1003], 95.00th=[ 1787],
| 99.00th=[ 2769], 99.50th=[ 3037], 99.90th=[ 3037], 99.95th=[ 3037],
| 99.99th=[ 3037]
bw ( KiB/s): min= 87, max= 2750, per=100.00%, avg=1665.82, stdev=222.17,
samples=159
iops : min= 20, max= 686, avg=415.20, stdev=55.54, samples=159
lat (msec) : 50=1.04%, 100=6.55%, 250=63.15%, 500=12.55%, 750=2.97%
lat (msec) : 1000=4.97%, 2000=5.37%, >=2000=4.76%
cpu : usr=0.08%, sys=0.29%, ctx=9014, majf=0, minf=60
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=0,9050,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
rand-read-qd32: (groupid=3, jobs=4): err= 0: pid=46538: Thu Sep 9 14:18:05 2021
read: IOPS=456k, BW=1780MiB/s (1867MB/s)(52.2GiB/30005msec)
slat (nsec): min=1610, max=18410k, avg=5101.31, stdev=87261.28
clat (nsec): min=683, max=263644k, avg=274516.14, stdev=1220977.05
lat (usec): min=2, max=263649, avg=279.84, stdev=1225.27
clat percentiles (nsec):
| 1.00th=[ 884], 5.00th=[ 5216], 10.00th=[ 12992],
| 20.00th=[ 28288], 30.00th=[ 45824], 40.00th=[ 67072],
| 50.00th=[ 89600], 60.00th=[ 112128], 70.00th=[ 134144],
| 80.00th=[ 162816], 90.00th=[ 222208], 95.00th=[ 937984],
| 99.00th=[ 4292608], 99.50th=[ 8093696], 99.90th=[10420224],
| 99.95th=[12124160], 99.99th=[14221312]
bw ( MiB/s): min= 865, max= 2219, per=98.89%, avg=1760.43, stdev=54.09,
samples=236
iops : min=221460, max=568171, avg=450667.59, stdev=13847.97,
samples=236
lat (nsec) : 750=0.01%, 1000=2.38%
lat (usec) : 2=0.73%, 4=0.01%, 10=5.33%, 20=6.17%, 50=17.40%
lat (usec) : 100=22.47%, 250=37.21%, 500=1.16%, 750=1.25%, 1000=1.06%
lat (msec) : 2=1.47%, 4=2.14%, 10=1.10%, 20=0.12%, 50=0.01%
lat (msec) : 500=0.01%
cpu : usr=26.98%, sys=44.88%, ctx=37869, majf=0, minf=60
IO depths : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued rwts: total=13674271,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
rand-write-qd1: (groupid=4, jobs=1): err= 0: pid=46552: Thu Sep 9 14:18:05 2021
write: IOPS=213, BW=854KiB/s (875kB/s)(25.0MiB/30003msec); 0 zone resets
slat (usec): min=14, max=10560, avg=38.97, stdev=184.01
clat (usec): min=599, max=2263.4k, avg=4638.59, stdev=59351.46
lat (usec): min=673, max=2270.0k, avg=4678.12, stdev=59482.51
clat percentiles (usec):
| 1.00th=[ 750], 5.00th=[ 824], 10.00th=[ 898],
| 20.00th=[ 2180], 30.00th=[ 2311], 40.00th=[ 2376],
| 50.00th=[ 2474], 60.00th=[ 2900], 70.00th=[ 3032],
| 80.00th=[ 3130], 90.00th=[ 3228], 95.00th=[ 3326],
| 99.00th=[ 3589], 99.50th=[ 4047], 99.90th=[ 265290],
| 99.95th=[1853883], 99.99th=[2264925]
bw ( KiB/s): min= 7, max= 1696, per=100.00%, avg=1249.83, stdev=622.40,
samples=41
iops : min= 1, max= 424, avg=312.44, stdev=155.64, samples=41
lat (usec) : 750=1.14%, 1000=13.53%
lat (msec) : 2=0.92%, 4=83.90%, 10=0.08%, 20=0.05%, 50=0.03%
lat (msec) : 100=0.09%, 250=0.14%, 500=0.03%, 2000=0.05%, >=2000=0.05%
cpu : usr=0.14%, sys=0.85%, ctx=6432, majf=0, minf=15
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,6409,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
rand-read-qd1: (groupid=5, jobs=1): err= 0: pid=46565: Thu Sep 9 14:18:05
2021
read: IOPS=20.4k, BW=79.8MiB/s (83.7MB/s)(2396MiB/30001msec)
slat (nsec): min=1509, max=747904, avg=3997.22, stdev=4935.97
clat (nsec): min=663, max=584288k, avg=44036.11, stdev=762638.81
lat (usec): min=2, max=584300, avg=48.18, stdev=763.33
clat percentiles (nsec):
| 1.00th=[ 676], 5.00th=[ 684], 10.00th=[ 692],
| 20.00th=[ 724], 30.00th=[ 740], 40.00th=[ 756],
| 50.00th=[ 780], 60.00th=[ 812], 70.00th=[ 972],
| 80.00th=[ 1704], 90.00th=[ 1864], 95.00th=[ 489472],
| 99.00th=[ 782336], 99.50th=[ 823296], 99.90th=[ 888832],
| 99.95th=[ 913408], 99.99th=[1019904]
bw ( KiB/s): min=29208, max=91600, per=99.93%, avg=81709.86, stdev=9983.04,
samples=59
iops : min= 7302, max=22900, avg=20427.44, stdev=2495.74, samples=59
lat (nsec) : 750=35.16%, 1000=35.73%
lat (usec) : 2=19.71%, 4=1.79%, 10=0.04%, 20=0.43%, 50=0.05%
lat (usec) : 100=0.01%, 250=0.01%, 500=2.24%, 750=3.41%, 1000=1.41%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 750=0.01%
cpu : usr=5.37%, sys=8.85%, ctx=43635, majf=0, minf=14
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=613263,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=58.8MiB/s (61.6MB/s), 58.8MiB/s-58.8MiB/s (61.6MB/s-61.6MB/s),
io=1792MiB (1879MB), run=30492-30492msec
Run status group 1 (all jobs):
READ: bw=5101MiB/s (5349MB/s), 5101MiB/s-5101MiB/s (5349MB/s-5349MB/s),
io=150GiB (161GB), run=30111-30111msec
Run status group 2 (all jobs):
WRITE: bw=1215KiB/s (1244kB/s), 1215KiB/s-1215KiB/s (1244kB/s-1244kB/s),
io=35.8MiB (37.6MB), run=30212-30212msec
Run status group 3 (all jobs):
READ: bw=1780MiB/s (1867MB/s), 1780MiB/s-1780MiB/s (1867MB/s-1867MB/s),
io=52.2GiB (56.0GB), run=30005-30005msec
Run status group 4 (all jobs):
WRITE: bw=854KiB/s (875kB/s), 854KiB/s-854KiB/s (875kB/s-875kB/s), io=25.0MiB
(26.3MB), run=30003-30003msec
Run status group 5 (all jobs):
READ: bw=79.8MiB/s (83.7MB/s), 79.8MiB/s-79.8MiB/s (83.7MB/s-83.7MB/s),
io=2396MiB (2512MB), run=30001-30001msec
Disk stats (read/write):
sda: ios=1308653/25511, merge=446/2810, ticks=4360315/7045149,
in_queue=11467004, util=97.78%_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/3YRR33PL7KA47NBKDEFYGQ36ECEPLD4V/
