Re: linux-aio usable?
On 03/08/2010 10:19 PM, Michael Tokarev wrote: Michael Tokarev wrote: [] Apparently that does not quite work. I just re-compiled kvm with --enable-linux-aio (actually I just installed libaio-dev on debian and qemu-kvm's configure picked it up automatically), and tried a guest. But any I/O fails. It has nothing to do with kvm. It is compat_ioctl32 in the kernel wrt aio calls. Historically I've a 64bit kernel with 32bit userland, and tried 32bit kvm too, and that does not work. But 64bit kvm works just fine with aio, and the performance numbers are indeed better. Can you elaborate? This sounds like a bug that wants to be fixed. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On 03/08/2010 11:27 PM, Nikola Ciprich wrote: It's faster. Hi Avi, Could You give some rough estimate on how much faster? The standard it depends on the workload. I'm stuck with glibc-2.5 now, but I'm always eager to improve performance, so I wonder if it would make sense to either port eventfd + aio stuff, or switch to glibc-2.8 for me... Switching to a modern setup should be much easier and safer. Esp. a modern kernel. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
Avi Kivity wrote:
On 03/08/2010 10:19 PM, Michael Tokarev wrote:
Michael Tokarev wrote:
[]
Apparently that does not quite work. I just re-compiled kvm with
--enable-linux-aio (actually I just installed libaio-dev on debian
and qemu-kvm's configure picked it up automatically), and tried
a guest. But any I/O fails.
It has nothing to do with kvm. It is compat_ioctl32 in the kernel
wrt aio calls. Historically I've a 64bit kernel with 32bit userland,
and tried 32bit kvm too, and that does not work. But 64bit kvm works
just fine with aio, and the performance numbers are indeed better.
Can you elaborate? This sounds like a bug that wants to be fixed.
http://thread.gmane.org/gmane.linux.kernel.aio.general/2891
It's missing compat_ioctl for some of the aio opcodes, namely
it's PREADV and PWRITE - the only ones used by kvm and the only
ones missing in kernel.
As far as i can see, current code converts the iocb array just
fine, but does not touch iovec array used with p{read,write}v.
/mjt
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On 03/09/2010 11:19 AM, Michael Tokarev wrote:
Can you elaborate? This sounds like a bug that wants to be fixed.
http://thread.gmane.org/gmane.linux.kernel.aio.general/2891
It's missing compat_ioctl for some of the aio opcodes, namely
it's PREADV and PWRITE - the only ones used by kvm and the only
ones missing in kernel.
As far as i can see, current code converts the iocb array just
fine, but does not touch iovec array used with p{read,write}v.
Yikes, looks pretty bad.
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On 03/08/2010 03:46 AM, Bernhard Schmidt wrote: Hi, sorry for this pretty generic question, I did not find any real pros and cons on the net anywhere, but I might just have missed them. In a pure x86_64 environment (~2.6.32 vanilla kernel, 0.12.3 qemu-kvm), is enabling linux-aio in KVM a good idea? Yes. What are the advantages/disadvantages? It's faster. Are there any potential pitfalls? It won't work well unless running on a block device (partition or LVM). The reason I'm asking is that there has been some traffic on the list about it, so it seems to be something people want to get working. qemu-kvm in Ubuntu Lucid is currently not compiled with that option. I've made a local version with aio and it seems to work fine (and performs a bit better at first glance). Is there any reason one should not compile that feature by default? Not to my knowledge. Does it do anything if not explicitly run with aio=native? IIUC, no. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On Mon, Mar 08, 2010 at 11:10:29AM +0200, Avi Kivity wrote: Are there any potential pitfalls? It won't work well unless running on a block device (partition or LVM). What does work well mean in this context? Potential dataloss? Is there any reason one should not compile that feature by default? Not to my knowledge. Thanks, I've filed a bug with Ubuntu to get it enabled. Bernhard -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On 03/08/2010 11:48 AM, Bernhard Schmidt wrote: On Mon, Mar 08, 2010 at 11:10:29AM +0200, Avi Kivity wrote: Are there any potential pitfalls? It won't work well unless running on a block device (partition or LVM). What does work well mean in this context? Potential dataloss? No, it becomes synchronous (=extra slow). -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On Mon, Mar 08, 2010 at 11:10:29AM +0200, Avi Kivity wrote: Are there any potential pitfalls? It won't work well unless running on a block device (partition or LVM). It will actually work well on pre-allocated filesystem images, at least on XFS and NFS. The real pitfal is that cache=none is required for kernel support as it only supports O_DIRECT. Is there any reason one should not compile that feature by default? It's compiled by default if libaio and it's development headers are found. Does it do anything if not explicitly run with aio=native? No. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On Mon, Mar 8, 2010 at 3:48 AM, Avi Kivity a...@redhat.com wrote: On 03/08/2010 11:48 AM, Bernhard Schmidt wrote: On Mon, Mar 08, 2010 at 11:10:29AM +0200, Avi Kivity wrote: Are there any potential pitfalls? It won't work well unless running on a block device (partition or LVM). What does work well mean in this context? Potential dataloss? No, it becomes synchronous (=extra slow). But for this to happen, the user would have had to consciously enter into the situation by creating/using a non block device, non-pre-allocated backing disk AND specify the aio=native option, correct? :-Dustin -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On 03/08/2010 04:25 PM, Dustin Kirkland wrote: On Mon, Mar 8, 2010 at 3:48 AM, Avi Kivitya...@redhat.com wrote: On 03/08/2010 11:48 AM, Bernhard Schmidt wrote: On Mon, Mar 08, 2010 at 11:10:29AM +0200, Avi Kivity wrote: Are there any potential pitfalls? It won't work well unless running on a block device (partition or LVM). What does work well mean in this context? Potential dataloss? No, it becomes synchronous (=extra slow). But for this to happen, the user would have had to consciously enter into the situation by creating/using a non block device, non-pre-allocated backing disk AND specify the aio=native option, correct? I thought there was some autodetection involved, but perhaps I just imagined it. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On 03/08/2010 08:26 AM, Avi Kivity wrote: On 03/08/2010 04:25 PM, Dustin Kirkland wrote: On Mon, Mar 8, 2010 at 3:48 AM, Avi Kivitya...@redhat.com wrote: On 03/08/2010 11:48 AM, Bernhard Schmidt wrote: On Mon, Mar 08, 2010 at 11:10:29AM +0200, Avi Kivity wrote: Are there any potential pitfalls? It won't work well unless running on a block device (partition or LVM). What does work well mean in this context? Potential dataloss? No, it becomes synchronous (=extra slow). But for this to happen, the user would have had to consciously enter into the situation by creating/using a non block device, non-pre-allocated backing disk AND specify the aio=native option, correct? I thought there was some autodetection involved, but perhaps I just imagined it. There's no autodetection. linux-aio support in the kernel downgrades to synchronous IO if the underlying storage does not support linux-aio. There is no indication to userspace that this has happened. If this happens, besides having a slow guest, the guest VCPU will be starved during the I/O requests potentially resulting in things like soft lockups and time drift. Generally, speaking, linux-aio will work well under the following circumstances: - cache=off is specified - the underlying file system is XFS or you are using a block device We cannot detect this reliably though so it's really up to the user to decide whether to use it. We're working on improving the linux-aio kernel interface though to eliminate this detectability problem after which, we can enable it in a more automatic fashion. Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On 03/08/2010 06:28 PM, Anthony Liguori wrote: I thought there was some autodetection involved, but perhaps I just imagined it. There's no autodetection. linux-aio support in the kernel downgrades to synchronous IO if the underlying storage does not support linux-aio. There is no indication to userspace that this has happened. If this happens, besides having a slow guest, the guest VCPU will be starved during the I/O requests potentially resulting in things like soft lockups and time drift. Generally, speaking, linux-aio will work well under the following circumstances: - cache=off is specified - the underlying file system is XFS or you are using a block device We cannot detect this reliably though so it's really up to the user to decide whether to use it. We're working on improving the linux-aio kernel interface though to eliminate this detectability problem after which, we can enable it in a more automatic fashion. Well, the common case of cache=none on a block device certainly can be autodetected. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
Avi Kivity wrote:
On 03/08/2010 03:46 AM, Bernhard Schmidt wrote:
Hi,
sorry for this pretty generic question, I did not find any real pros and
cons on the net anywhere, but I might just have missed them.
In a pure x86_64 environment (~2.6.32 vanilla kernel, 0.12.3 qemu-kvm),
is enabling linux-aio in KVM a good idea?
Yes.
Apparently that does not quite work. I just re-compiled kvm with
--enable-linux-aio (actually I just installed libaio-dev on debian
and qemu-kvm's configure picked it up automatically), and tried
a guest. But any I/O fails.
kvm-0.12.3 ... -drive file=/dev/sda10,if=virtio,cache=none,aio=native
(/dev/sda10 is a (spare) partition on my hard drive I use for testing).
Here's the resulting dmesg in the guest (2.6.32):
vdb:
end_request: I/O error, dev vdb, sector 0
Buffer I/O error on device vdb, logical block 0
Buffer I/O error on device vdb, logical block 1
Buffer I/O error on device vdb, logical block 2
Buffer I/O error on device vdb, logical block 3
Buffer I/O error on device vdb, logical block 4
Buffer I/O error on device vdb, logical block 5
Buffer I/O error on device vdb, logical block 6
Buffer I/O error on device vdb, logical block 7
end_request: I/O error, dev vdb, sector 0
Buffer I/O error on device vdb, logical block 0
Buffer I/O error on device vdb, logical block 1
end_request: I/O error, dev vdb, sector 0
unable to read partition table
And any I/O - be it reads of writes - fails.
I see some aio_submit() etc are happening in strace,
but no errors. Unfortunately my strace does not
decode io_*() routines.
# fgrep io_ trc
...
1227 io_submit(4152147968, 1, {...}) = 1
1226 io_getevents(-142819328, 1, 128, {...}{0, 0}) = 1
1227 io_submit(4152147968, 1, {...}) = 1
1226 io_getevents(-142819328, 1, 128, {...}{0, 0}) = 1
1227 io_submit(4152147968, 1, {...}) = 1
1226 io_getevents(-142819328, 1, 128, {...}{0, 0}) = 1
...
Oh well ;)
/mjt
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
Michael Tokarev wrote: [] Apparently that does not quite work. I just re-compiled kvm with --enable-linux-aio (actually I just installed libaio-dev on debian and qemu-kvm's configure picked it up automatically), and tried a guest. But any I/O fails. It has nothing to do with kvm. It is compat_ioctl32 in the kernel wrt aio calls. Historically I've a 64bit kernel with 32bit userland, and tried 32bit kvm too, and that does not work. But 64bit kvm works just fine with aio, and the performance numbers are indeed better. Thanks! /mjt -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
It's faster. Hi Avi, Could You give some rough estimate on how much faster? I'm stuck with glibc-2.5 now, but I'm always eager to improve performance, so I wonder if it would make sense to either port eventfd + aio stuff, or switch to glibc-2.8 for me... -- - Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax:+420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: ser...@linuxbox.cz - -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux-aio usable?
On Monday 08 March 2010 03:27:36 pm Nikola Ciprich wrote: It's faster. Hi Avi, Could You give some rough estimate on how much faster? I'm stuck with glibc-2.5 now, but I'm always eager to improve performance, so I wonder if it would make sense to either port eventfd + aio stuff, or switch to glibc-2.8 for me... I saw approx. 10% improvement in sequential i/o. Random i/o was only marginally faster in our setup. We generally have problems with random i/o here... Something to do with our setup. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
linux-aio usable?
Hi, sorry for this pretty generic question, I did not find any real pros and cons on the net anywhere, but I might just have missed them. In a pure x86_64 environment (~2.6.32 vanilla kernel, 0.12.3 qemu-kvm), is enabling linux-aio in KVM a good idea? What are the advantages/disadvantages? Are there any potential pitfalls? The reason I'm asking is that there has been some traffic on the list about it, so it seems to be something people want to get working. qemu-kvm in Ubuntu Lucid is currently not compiled with that option. I've made a local version with aio and it seems to work fine (and performs a bit better at first glance). Is there any reason one should not compile that feature by default? Does it do anything if not explicitly run with aio=native? Best Regards, Bernhard -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
