Hello, Steffen, Thanks for Your reply. Sorry but I was on holidays, now I'm back and still digging into my problem.. :(
I've read thousands of google links but can't find anything which could help me. - tried all qemu drive IO(io=) and cache(cache=) modes, nothing could come even close to the results I'm getting outside KVM. - enabling blk-mq inside KVM guest didn't help - enabling iothreads didn't help - 'queues' parameter in my libvirt schemas can be only applied to 'virtio-serial', can't use it in virtio-scsi nor virtio-blk) - I've seen some people using 'num_queues' but I don't have this parameter in my schemas(libvirt version = 1.3.1, qemu version = 2.5.0) So, is there really no way to increase queue_depth to rbd device in kvm domain or any other way to achieve results similar to those obtained outside KVM? :/ wt., 26 cze 2018 o 15:19 Steffen Winther Sørensen <[email protected]> napisał(a): > > > > On 26 Jun 2018, at 14.04, Damian Dabrowski <[email protected]> wrote: > > > > Hi Stefan, thanks for reply. > > > > Unfortunately it didn't work. > > > > disk config: > > <disk type='network' device='disk'> > > <driver name='qemu' type='raw' cache='directsync' io='native' > discard='unmap'/> > > <auth username='compute'> > > <secret type='ceph' uuid='a5d0dd94-57c4-ae55-ffe0-7e3732a24455'/> > > </auth> > > <source protocol='rbd' > name='volumes-nvme/volume-ce247187-a625-49f1-bacd-fc03df215395'> > > <host name='172.31.0.1' port='6789'/> > > <host name='172.31.0.2' port='6789'/> > > <host name='172.31.0.3' port='6789'/> > > </source> > > <target dev='sdb' bus='scsi'/> > > <serial>ce247187-a625-49f1-bacd-fc03df215395</serial> > > <address type='drive' controller='0' bus='0' target='0' unit='2'/> > > </disk> > > > > > > Controller config: > > <controller type='scsi' index='0' model='virtio-scsi'> > > <driver queues='8'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x0'/> > > </controller> > > > > > > benchmark command: fio --randrepeat=1 --ioengine=libaio --direct=1 > --name=test --filename=test --bs=4k --iodepth=64 --size=1G > --readwrite=randwrite --time_based --runtime=60 > --write_iops_log=write_results --numjobs=8 > > > > And I'm still getting very low random write IOPS inside KVM instance > with 8vcores(3-5k compared to 20k+ outside KVM) > > > > Maybe do You have any idea how to deal with it? > What about trying with io=‘threads’ and/or maybe cache=’none’ or swap from > virtio-scsi to blk-mq? > > Other people had similar issues, try asking ‘G’ > > https://serverfault.com/questions/425607/kvm-guest-io-is-much-slower-than-host-io-is-that-normal > https://wiki.mikejung.biz/KVM_/_Xen > > /Steffen > >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
