Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-19 Thread Michael S. Tsirkin
On Thu, Jun 16, 2011 at 11:01:22PM +0800, Asias He wrote: On 06/16/2011 10:28 PM, Michael S. Tsirkin wrote: On Wed, Jun 15, 2011 at 06:53:34PM +0300, Pekka Enberg wrote: Hi all, We’re proud to announce the second version of the Native Linux KVM tool! We’re now officially aiming for

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-17 Thread Jeff Garzik
On 06/16/2011 07:22 AM, Ingo Molnar wrote: * Christoph Hellwigh...@infradead.org wrote: On Thu, Jun 16, 2011 at 12:57:36PM +0300, Pekka Enberg wrote: Uh-oh. Someone needs to apply this patch to sync_file_range(): There actually are a few cases where using it makes sense. [...] Such as?

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-17 Thread justin
On 2011年06月15日 23:53, Pekka Enberg wrote: or alternatively, if you already have a kernel source tree: git remote add kvm-tool git://github.com/penberg/linux-kvm.git git remote update git checkout -b kvm-tool/master kvm-tool I tried this, but it do not work, there is something wrong

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-17 Thread Sasha Levin
On Fri, 2011-06-17 at 06:00 +0100, Stefan Hajnoczi wrote: On Fri, Jun 17, 2011 at 2:03 AM, Sasha Levin levinsasha...@gmail.com wrote: On Thu, 2011-06-16 at 17:50 -0500, Anthony Liguori wrote: On 06/16/2011 09:48 AM, Pekka Enberg wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-17 Thread Anthony Liguori
On 06/16/2011 08:03 PM, Sasha Levin wrote: On Thu, 2011-06-16 at 17:50 -0500, Anthony Liguori wrote: Each virtio-blk device can process requests regardless of other virtio-blk devices, which means that we can do parallel requests for devices. Within each device, we support parallel requests in

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Pekka Enberg
On Thu, Jun 16, 2011 at 1:44 AM, Anthony Liguori anth...@codemonkey.ws wrote: That's probably why it's fast, it doesn't preserve data integrity :( Actually, I misread the code.  It does unstable writes but it does do fsync() on FLUSH. On Thu, Jun 16, 2011 at 8:41 AM, Pekka Enberg

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Ingo Molnar
* Pekka Enberg penb...@kernel.org wrote: On Thu, Jun 16, 2011 at 1:07 AM, Alexander Graf ag...@suse.de wrote: qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO? We don't

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Pekka Enberg
Hi Ingo, On Thu, Jun 16, 2011 at 10:24 AM, Ingo Molnar mi...@elte.hu wrote:  - executing AIO in the vcpu thread eats up precious vcpu execution   time: combined QCOW2 throughput would be limited by a single   core's performance, and any time spent on QCOW2 processing would   not be spent

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Ingo Molnar
* Pekka Enberg penb...@kernel.org wrote: Hi Ingo, On Thu, Jun 16, 2011 at 10:24 AM, Ingo Molnar mi...@elte.hu wrote:  - executing AIO in the vcpu thread eats up precious vcpu execution   time: combined QCOW2 throughput would be limited by a single   core's performance, and any time

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Stefan Hajnoczi
On Thu, Jun 16, 2011 at 8:24 AM, Ingo Molnar mi...@elte.hu wrote:  - executing AIO in the vcpu thread eats up precious vcpu execution   time: combined QCOW2 throughput would be limited by a single   core's performance, and any time spent on QCOW2 processing would   not be spent running the

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Christoph Hellwig
On Thu, Jun 16, 2011 at 09:21:03AM +0300, Pekka Enberg wrote: And btw, we use sync_file_range() Which doesn't help you at all. sync_file_range is just a hint for VM writeback, but never commits filesystem metadata nor the physical disk's write cache. In short it's a completely dangerous

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Pekka Enberg
Hi Christoph, On Thu, Jun 16, 2011 at 09:21:03AM +0300, Pekka Enberg wrote: And btw, we use sync_file_range() On Thu, Jun 16, 2011 at 12:24 PM, Christoph Hellwig h...@infradead.org wrote: Which doesn't help you at all.  sync_file_range is just a hint for VM writeback, but never commits

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Christoph Hellwig
On Thu, Jun 16, 2011 at 12:34:04PM +0300, Pekka Enberg wrote: Hi Christoph, On Thu, Jun 16, 2011 at 09:21:03AM +0300, Pekka Enberg wrote: And btw, we use sync_file_range() On Thu, Jun 16, 2011 at 12:24 PM, Christoph Hellwig h...@infradead.org wrote: Which doesn't help you at all.

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Ingo Molnar
* Christoph Hellwig h...@infradead.org wrote: What's the right thing to do here? Is fdatasync() sufficient? Yes. Prasad, Pekka, mind redoing the numbers with fdatasync()? I'd be surprised if they were significantly worse but it has to be done to have apples-to-apples numbers. Thanks,

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Pekka Enberg
On Thu, Jun 16, 2011 at 12:48 PM, Christoph Hellwig h...@infradead.org wrote: You also missed: This system call does not flush disk write caches and thus does not  provide any data integrity on systems with volatile disk write  caches. so it's not safe if you either have a cache, or are

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Christoph Hellwig
On Thu, Jun 16, 2011 at 12:57:36PM +0300, Pekka Enberg wrote: Uh-oh. Someone needs to apply this patch to sync_file_range(): There actually are a few cases where using it makes sense. It's just the minority. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Ingo Molnar
* Christoph Hellwig h...@infradead.org wrote: On Thu, Jun 16, 2011 at 12:57:36PM +0300, Pekka Enberg wrote: Uh-oh. Someone needs to apply this patch to sync_file_range(): There actually are a few cases where using it makes sense. [...] Such as? I don't think apps can actually know whether

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Christoph Hellwig
On Thu, Jun 16, 2011 at 01:22:30PM +0200, Ingo Molnar wrote: Such as? I don't think apps can actually know whether disk blocks have been 'instantiated' by a particular filesystem or not, so the manpage: In general they can't. The only good use case for sync_file_range is to paper

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Ingo Molnar
* Christoph Hellwig h...@infradead.org wrote: On Thu, Jun 16, 2011 at 01:22:30PM +0200, Ingo Molnar wrote: Such as? I don't think apps can actually know whether disk blocks have been 'instantiated' by a particular filesystem or not, so the manpage: In general they can't. The only

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Christoph Hellwig
On Thu, Jun 16, 2011 at 01:40:45PM +0200, Ingo Molnar wrote: Filesystems that cannot guarantee that should map their sync_file_range() implementation to fdatasync() or so, right? Filesystems aren't even told about sync_file_range, it's purely a VM thing, which is the root of the problem.

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Michael S. Tsirkin
On Wed, Jun 15, 2011 at 06:53:34PM +0300, Pekka Enberg wrote: Hi all, We’re proud to announce the second version of the Native Linux KVM tool! We’re now officially aiming for merging to mainline in 3.1. Highlights: - Experimental GUI support using SDL and VNC - SMP support.

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Pekka Enberg
On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enberg penb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the  following URL for test result details: https://gist.github.com/1026888 It turns out we were benchmarking the wrong guest kernel version for

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Asias He
On 06/16/2011 10:28 PM, Michael S. Tsirkin wrote: On Wed, Jun 15, 2011 at 06:53:34PM +0300, Pekka Enberg wrote: Hi all, We’re proud to announce the second version of the Native Linux KVM tool! We’re now officially aiming for merging to mainline in 3.1. Highlights: - Experimental GUI

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Anthony Liguori
On 06/16/2011 09:48 AM, Pekka Enberg wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enbergpenb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 It turns out we were

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Sasha Levin
On Thu, 2011-06-16 at 17:50 -0500, Anthony Liguori wrote: On 06/16/2011 09:48 AM, Pekka Enberg wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enbergpenb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Stefan Hajnoczi
On Fri, Jun 17, 2011 at 2:03 AM, Sasha Levin levinsasha...@gmail.com wrote: On Thu, 2011-06-16 at 17:50 -0500, Anthony Liguori wrote: On 06/16/2011 09:48 AM, Pekka Enberg wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enbergpenb...@kernel.org  wrote: - Fast QCOW2 image read-write support

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-16 Thread Stefan Hajnoczi
On Thu, Jun 16, 2011 at 3:48 PM, Pekka Enberg penb...@kernel.org wrote: On Wed, Jun 15, 2011 at 6:53 PM, Pekka Enberg penb...@kernel.org wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the  following URL for test result details: https://gist.github.com/1026888

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Avi Kivity
On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the following URL for test result details: https://gist.github.com/1026888 This is surprising. How is qemu invoked? btw the dump above is a little hard to interpret. --

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Pekka Enberg
On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivity a...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the   following URL for test result details: https://gist.github.com/1026888 This is surprising.  How is

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Prasad Joshi
On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enberg penb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivity a...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See the   following URL for test result

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Sasha Levin
On Wed, 2011-06-15 at 21:13 +0100, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enberg penb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivity a...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Prasad Joshi
On Wed, Jun 15, 2011 at 9:23 PM, Sasha Levin levinsasha...@gmail.com wrote: On Wed, 2011-06-15 at 21:13 +0100, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enberg penb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivity a...@redhat.com wrote: On 06/15/2011 06:53

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Anthony Liguori
On 06/15/2011 10:53 AM, Pekka Enberg wrote: Hi all, We’re proud to announce the second version of the Native Linux KVM tool! We’re now officially aiming for merging to mainline in 3.1. Highlights: - Experimental GUI support using SDL and VNC - SMP support. tools/kvm/ now has a highly

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Anthony Liguori
On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2 image read-write support beating Qemu in fio

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Alexander Graf
On 16.06.2011, at 00:04, Anthony Liguori wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com wrote: On 06/15/2011 06:53 PM, Pekka Enberg wrote: - Fast QCOW2

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Anthony Liguori
On 06/15/2011 05:07 PM, Alexander Graf wrote: On 16.06.2011, at 00:04, Anthony Liguori wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com wrote: On

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Anthony Liguori
On 06/15/2011 05:20 PM, Anthony Liguori wrote: On 06/15/2011 05:07 PM, Alexander Graf wrote: On 16.06.2011, at 00:04, Anthony Liguori wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org wrote: On Wed, Jun 15, 2011 at 7:30 PM,

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Stefan Hajnoczi
On Wed, Jun 15, 2011 at 11:04 PM, Anthony Liguori anth...@codemonkey.ws wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org  wrote: On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivitya...@redhat.com  wrote: On 06/15/2011 06:53 PM, Pekka

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Pekka Enberg
On Thu, Jun 16, 2011 at 1:44 AM, Anthony Liguori anth...@codemonkey.ws wrote: That's probably why it's fast, it doesn't preserve data integrity :( Actually, I misread the code.  It does unstable writes but it does do fsync() on FLUSH. Yes. That's fine, right? Or did we misread how virtio

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Pekka Enberg
On Thu, Jun 16, 2011 at 8:29 AM, Stefan Hajnoczi stefa...@gmail.com wrote: On Wed, Jun 15, 2011 at 11:04 PM, Anthony Liguori anth...@codemonkey.ws wrote: On 06/15/2011 03:13 PM, Prasad Joshi wrote: On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enbergpenb...@kernel.org  wrote: On Wed, Jun 15, 2011

Re: [ANNOUNCE] Native Linux KVM tool v2

2011-06-15 Thread Pekka Enberg
On Thu, Jun 16, 2011 at 1:07 AM, Alexander Graf ag...@suse.de wrote: qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO? We don't use AIO at all. It's just normal read()/write() with