Hi,
I test in VM with fio, here is the config:
[global]
direct=1
ioengine=aio
iodepth=1
[sequence read 4K]
rw=read
bs=4K
size=1024m
directory=/mnt
filename=test
sequence read 4K: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio,
iodepth=1
fio-2.1.3
Starting 1 process
sequence read 4K: Laying out IO file(s) (1 file(s) / 1024MB)
^Cbs: 1 (f=1): [R] [18.0% done] [1994KB/0KB/0KB /s] [498/0/0 iops] [eta
07m:14s]
fio: terminating on signal 2
sequence read 4K: (groupid=0, jobs=1): err= 0: pid=1156: Fri Nov 21
12:32:53 2014
read : io=187408KB, bw=1984.1KB/s, iops=496, runt= 94417msec
slat (usec): min=22, max=878, avg=48.36, stdev=22.63
clat (usec): min=1335, max=17618, avg=1956.45, stdev=247.26
lat (usec): min=1371, max=17680, avg=2006.97, stdev=248.47
clat percentiles (usec):
| 1.00th=[ 1560], 5.00th=[ 1640], 10.00th=[ 1704], 20.00th=[ 1784],
| 30.00th=[ 1848], 40.00th=[ 1896], 50.00th=[ 1944], 60.00th=[ 1992],
| 70.00th=[ 2064], 80.00th=[ 2128], 90.00th=[ 2192], 95.00th=[ 2288],
| 99.00th=[ 2448], 99.50th=[ 2640], 99.90th=[ 3856], 99.95th=[ 4256],
| 99.99th=[ 9408]
bw (KB /s): min= 1772, max= 2248, per=100.00%, avg=1986.55,
stdev=85.76
lat (msec) : 2=60.69%, 4=39.23%, 10=0.07%, 20=0.01%
cpu : usr=1.92%, sys=2.98%, ctx=47125, majf=0, minf=28
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued : total=r=46852/w=0/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=187408KB, aggrb=1984KB/s, minb=1984KB/s, maxb=1984KB/s,
mint=94417msec, maxt=94417msec
Disk stats (read/write):
sda: ios=46754/11, merge=0/10, ticks=91144/40, in_queue=91124,
util=96.73%
the
the rados benchmark:
# rados -p volumes bench 60 seq -b 4096 -t 1
Total time run: 44.922178
Total reads made: 24507
Read size: 4096
Bandwidth (MB/sec): 2.131
Average Latency: 0.00183069
Max latency: 0.004598
Min latency: 0.001224
Re: [ceph-users] RBD read-ahead didn't improve 4K read performance
Alexandre DERUMIER
收件人:
duan xufeng
2014/11/21 14:51
抄送:
si dawei, ceph-users
Hi,
I don't have tested yet rbd readhead,
but maybe do you reach qemu limit. (by default qemu can use only
1thread/1core to manage ios, check you qemu cpu).
Do you have some performance results ? how many iops ?
but I have had 4x improvement in qemu-kvm, with virtio-scsi + num_queues +
lasts kernel.
(4k seq coalesced reads in qemu, was doing bigger iops to ceph).
libvirt : <controller type='scsi' index='0' model='virtio-scsi'
num_queues='8'/>
Regards,
Alexandre
----- Mail original -----
De: "duan xufeng" <[email protected]>
À: "ceph-users" <[email protected]>
Cc: "si dawei" <[email protected]>
Envoyé: Vendredi 21 Novembre 2014 03:58:38
Objet: [ceph-users] RBD read-ahead didn't improve 4K read performance
hi，
I upgraded CEPH to 0.87 for rbd readahead , but can't see any performance
improvement in 4K seq read in the VM.
How can I know if the readahead is take effect?
thanks.
ceph.conf
[client]
rbd_cache = true
rbd_cache_size = 335544320
rbd_cache_max_dirty = 251658240
rbd_cache_target_dirty = 167772160
rbd readahead trigger requests = 1
rbd readahead max bytes = 4194304
rbd readahead disable after bytes = 0
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail
(and any attachment transmitted herewith) is privileged and confidential
and is intended for the exclusive use of the addressee(s). If you are not
an intended recipient, any disclosure, reproduction, distribution or other
dissemination or use of the information contained is strictly prohibited.
If you have received this mail in error, please delete it and notify us
immediately.
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and
any attachment transmitted herewith) is privileged and confidential and is
intended for the exclusive use of the addressee(s). If you are not an intended
recipient, any disclosure, reproduction, distribution or other dissemination or
use of the information contained is strictly prohibited. If you have received
this mail in error, please delete it and notify us immediately.
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
