Re: [ceph-users] large reads become 512 kbyte reads on qemu-kvm rbd

2014-12-01 Thread Ilya Dryomov
On Mon, Dec 1, 2014 at 1:09 PM, Dan Van Der Ster daniel.vanders...@cern.ch wrote: Hi Ilya, On 28 Nov 2014, at 17:56, Ilya Dryomov ilya.dryo...@inktank.com wrote: On Fri, Nov 28, 2014 at 5:46 PM, Dan Van Der Ster daniel.vanders...@cern.ch wrote: Hi Andrei, Yes, I’m testing from within the

Re: [ceph-users] large reads become 512 kbyte reads on qemu-kvm rbd

2014-11-28 Thread Dan Van Der Ster
Hi, After some more tests we’ve found that max_sectors_kb is the reason for splitting large IOs. We increased it to 4MB: echo 4096 cat /sys/block/vdb/queue/max_sectors_kb and now fio/iostat are showing reads up to 4MB are getting through to the block device unsplit. We use 4MB to match the

Re: [ceph-users] large reads become 512 kbyte reads on qemu-kvm rbd

2014-11-28 Thread Andrei Mikhailovsky
November, 2014 1:33:20 PM Subject: Re: [ceph-users] large reads become 512 kbyte reads on qemu-kvm rbd Hi, After some more tests we’ve found that max_sectors_kb is the reason for splitting large IOs. We increased it to 4MB: echo 4096 cat /sys/block/vdb/queue/max_sectors_kb and now fio

Re: [ceph-users] large reads become 512 kbyte reads on qemu-kvm rbd

2014-11-28 Thread Dan Van Der Ster
:20 PM Subject: Re: [ceph-users] large reads become 512 kbyte reads on qemu-kvm rbd Hi, After some more tests we’ve found that max_sectors_kb is the reason for splitting large IOs. We increased it to 4MB: echo 4096 cat /sys/block/vdb/queue/max_sectors_kb and now fio/iostat are showing reads up

Re: [ceph-users] large reads become 512 kbyte reads on qemu-kvm rbd

2014-11-28 Thread Ilya Dryomov
On Fri, Nov 28, 2014 at 5:46 PM, Dan Van Der Ster daniel.vanders...@cern.ch wrote: Hi Andrei, Yes, I’m testing from within the guest. Here is an example. First, I do 2MB reads when the max_sectors_kb=512, and we see the reads are split into 4. (fio sees 25 iops, though iostat reports 100

Re: [ceph-users] large reads become 512 kbyte reads on qemu-kvm rbd

2014-11-28 Thread Lindsay Mathieson
On Fri, 28 Nov 2014 08:56:24 PM Ilya Dryomov wrote: which you are supposed to change on a per-device basis via sysfs. Is there a way to do this for windows VM's? -- Lindsay signature.asc Description: This is a digitally signed message part. ___

Re: [ceph-users] large reads become 512 kbyte reads on qemu-kvm rbd

2014-11-27 Thread Dan Van Der Ster
Oops, I was off by a factor of 1000 in my original subject. We actually have 4M and 8M reads being split into 100 512kB reads per second. So perhaps these are limiting: # cat /sys/block/vdb/queue/max_sectors_kb 512 # cat /sys/block/vdb/queue/read_ahead_kb 512 Questions below remain. Cheers,