And more information:
I test an fio job, I got the settings not by my own instaed someone
from the forum give me this for testing ZFS:
root@ph-pbs:~# fio --name=typical-vm --size=8G --rw=readwrite
--rwmixread=69 --direct=1 --bs=4K --numjobs=4 --ioengine=libaio
--iodepth=12 --group_reporting --runtime=20m --time_based^C
root@ph-pbs:~# cd /ZFSPOOL/
BACKUPSTORE001/ VMSTORE001/
root@ph-pbs:~# cd /ZFSPOOL/VMSTORE001/
root@ph-pbs:/ZFSPOOL/VMSTORE001# fio --name=typical-vm --size=8G
--rw=readwrite --rwmixread=69 --direct=1 --bs=4K --numjobs=4
--ioengine=libaio --iodepth=12 --group_reporting --runtime=20m
--time_based
typical-vm: (g=0): rw=rw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=12
...
fio-3.25
Starting 4 processes
typical-vm: Laying out IO file (1 file / 8192MiB)
typical-vm: Laying out IO file (1 file / 8192MiB)
typical-vm: Laying out IO file (1 file / 8192MiB)
typical-vm: Laying out IO file (1 file / 8192MiB)
Jobs: 4 (f=0): [f(4)][100.0%][r=1518MiB/s,w=682MiB/s][r=389k,w=175k
IOPS][eta 00m:00s]
typical-vm: (groupid=0, jobs=4): err= 0: pid=3804786: Fri Oct 1 10:56:30 2021
read: IOPS=356k, BW=1392MiB/s (1460MB/s)(1631GiB/1200001msec)
slat (nsec): min=1854, max=176589k, avg=5156.08, stdev=39010.68
clat (usec): min=4, max=191637, avg=85.89, stdev=133.21
lat (usec): min=32, max=191640, avg=91.13, stdev=139.42
clat percentiles (usec):
| 1.00th=[ 42], 5.00th=[ 46], 10.00th=[ 49], 20.00th=[ 52],
| 30.00th=[ 56], 40.00th=[ 59], 50.00th=[ 65], 60.00th=[ 85],
| 70.00th=[ 97], 80.00th=[ 111], 90.00th=[ 141], 95.00th=[ 176],
| 99.00th=[ 265], 99.50th=[ 318], 99.90th=[ 570], 99.95th=[ 693],
| 99.99th=[ 1090]
bw ( MiB/s): min= 250, max= 2159, per=100.00%, avg=1392.77,
stdev=63.78, samples=9596
iops : min=64218, max=552858, avg=356548.75,
stdev=16328.20, samples=9596
write: IOPS=160k, BW=626MiB/s (656MB/s)(733GiB/1200001msec); 0 zone resets
slat (usec): min=3, max=191425, avg= 9.71, stdev=34.41
clat (usec): min=2, max=191641, avg=86.02, stdev=137.32
lat (usec): min=35, max=191650, avg=95.85, stdev=144.10
clat percentiles (usec):
| 1.00th=[ 42], 5.00th=[ 46], 10.00th=[ 49], 20.00th=[ 52],
| 30.00th=[ 56], 40.00th=[ 59], 50.00th=[ 65], 60.00th=[ 85],
| 70.00th=[ 98], 80.00th=[ 111], 90.00th=[ 141], 95.00th=[ 178],
| 99.00th=[ 265], 99.50th=[ 318], 99.90th=[ 578], 99.95th=[ 701],
| 99.99th=[ 1106]
bw ( KiB/s): min=114464, max=995856, per=100.00%, avg=640817.51,
stdev=29342.79, samples=9596
iops : min=28616, max=248964, avg=160204.26, stdev=7335.70,
samples=9596
lat (usec) : 4=0.01%, 10=0.01%, 50=13.69%, 100=58.80%, 250=26.29%
lat (usec) : 500=1.08%, 750=0.10%, 1000=0.02%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%, 250=0.01%
cpu : usr=18.17%, sys=79.17%, ctx=982498, majf=10, minf=2977
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued rwts: total=427672030,192161509,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=12
Run status group 0 (all jobs):
READ: bw=1392MiB/s (1460MB/s), 1392MiB/s-1392MiB/s
(1460MB/s-1460MB/s), io=1631GiB (1752GB), run=1200001-1200001msec
WRITE: bw=626MiB/s (656MB/s), 626MiB/s-626MiB/s (656MB/s-656MB/s),
io=733GiB (787GB), run=1200001-1200001msec
And this is while two of the restore jobs still running.
_______________________________________________
pve-user mailing list
pve-user@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
