Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Avi Kivity
On 09/07/2010 04:41 PM, Anthony Liguori wrote: Hi, We've got copy-on-read and image streaming working in QED and before going much further, I wanted to bounce some interfaces off of the libvirt folks to make sure our final interface makes sense. Here's the basic idea: Today, you can

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Avi Kivity
On 09/07/2010 05:57 PM, Anthony Liguori wrote: I agree that streaming should be generic, like block migration. The trivial generic implementation is: void bdrv_stream(BlockDriverState* bs) { for (sector = 0; sector bdrv_getlength(bs); sector += n) { if (!bdrv_is_allocated(bs,

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Anthony Liguori
On 09/12/2010 07:41 AM, Avi Kivity wrote: On 09/07/2010 05:57 PM, Anthony Liguori wrote: I agree that streaming should be generic, like block migration. The trivial generic implementation is: void bdrv_stream(BlockDriverState* bs) { for (sector = 0; sector bdrv_getlength(bs); sector +=

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Avi Kivity
On 09/12/2010 03:25 PM, Anthony Liguori wrote: On 09/12/2010 07:41 AM, Avi Kivity wrote: On 09/07/2010 05:57 PM, Anthony Liguori wrote: I agree that streaming should be generic, like block migration. The trivial generic implementation is: void bdrv_stream(BlockDriverState* bs) { for

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Anthony Liguori
On 09/12/2010 08:40 AM, Avi Kivity wrote: Why would it serialize all I/O operations? It's just like another vcpu issuing reads. Because the block layer isn't re-entrant. What you basically do is: stream_step_three(): complete() stream_step_two(offset, length): bdrv_aio_readv(offset,

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Anthony Liguori
On 09/12/2010 11:45 AM, Avi Kivity wrote: Streaming relies on copy-on-read to do the writing. Ah. You can avoid the copy-on-read implementation in the block format driver and do it completely in generic code. Copy on read takes advantage of temporal locality. You wouldn't want to stream

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Avi Kivity
On 09/12/2010 07:19 PM, Anthony Liguori wrote: On 09/12/2010 11:45 AM, Avi Kivity wrote: Streaming relies on copy-on-read to do the writing. Ah. You can avoid the copy-on-read implementation in the block format driver and do it completely in generic code. Copy on read takes advantage of

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-08 Thread Kevin Wolf
Am 07.09.2010 17:09, schrieb Stefan Hajnoczi: Right, I'm a little hesitant to get too far into discussing the management interface because I remember long threads about polling and async. I never fully read them but I bet some wisdom came out of them that applies here. There are two ways

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:01 AM, Alexander Graf wrote: I'm torn here too. Why not expose both? Have a qemu internal daemon available that gets a sleep time as parameter and an external pull sectors command. We'll see which one is more useful, but I don't think it's too much code to justify only having

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:34 AM, Kevin Wolf wrote: Am 07.09.2010 15:41, schrieb Anthony Liguori: Hi, We've got copy-on-read and image streaming working in QED and before going much further, I wanted to bounce some interfaces off of the libvirt folks to make sure our final interface makes sense.

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:33 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 2:41 PM, Anthony Liguori aligu...@linux.vnet.ibm.com wrote: The interface for copy-on-read is just an option within qemu-img create. Streaming, on the other hand, requires a bit more thought. Today, I have a monitor

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:49 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:34 PM, Kevin Wolfkw...@redhat.com wrote: Am 07.09.2010 15:41, schrieb Anthony Liguori: Hi, We've got copy-on-read and image streaming working in QED and before going much further, I wanted to bounce some

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 2:41 PM, Anthony Liguori aligu...@linux.vnet.ibm.com wrote: The interface for copy-on-read is just an option within qemu-img create.  Streaming, on the other hand, requires a bit more thought.  Today, I have a monitor command that does the following: stream device

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 3:34 PM, Kevin Wolf kw...@redhat.com wrote: Am 07.09.2010 15:41, schrieb Anthony Liguori: Hi, We've got copy-on-read and image streaming working in QED and before going much further, I wanted to bounce some interfaces off of the libvirt folks to make sure our final

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Kevin Wolf
Am 07.09.2010 15:41, schrieb Anthony Liguori: Hi, We've got copy-on-read and image streaming working in QED and before going much further, I wanted to bounce some interfaces off of the libvirt folks to make sure our final interface makes sense. Here's the basic idea: Today, you can

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 3:51 PM, Anthony Liguori aligu...@linux.vnet.ibm.com wrote: On 09/07/2010 09:33 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 2:41 PM, Anthony Liguori aligu...@linux.vnet.ibm.com  wrote: The interface for copy-on-read is just an option within qemu-img create.  

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Kevin Wolf
Am 07.09.2010 16:49, schrieb Anthony Liguori: Shouldn't it be a runtime option? You can use the very same image with copy-on-read or copy-on-write and it will behave the same (execpt for performance), so it's not an inherent feature of the image file. The way it's implemented in QED is

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:55 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:51 PM, Anthony Liguori aligu...@linux.vnet.ibm.com wrote: On 09/07/2010 09:33 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 2:41 PM, Anthony Liguori aligu...@linux.vnet.ibm.comwrote: The

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 3:57 PM, Anthony Liguori aligu...@linux.vnet.ibm.com wrote: On 09/07/2010 09:49 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:34 PM, Kevin Wolfkw...@redhat.com  wrote: Am 07.09.2010 15:41, schrieb Anthony Liguori: Hi, We've got copy-on-read and image

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:02 AM, Kevin Wolf wrote: Am 07.09.2010 16:49, schrieb Anthony Liguori: Shouldn't it be a runtime option? You can use the very same image with copy-on-read or copy-on-write and it will behave the same (execpt for performance), so it's not an inherent feature of the image

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 4:00 PM, Anthony Liguori aligu...@linux.vnet.ibm.com wrote: On 09/07/2010 09:55 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:51 PM, Anthony Liguori aligu...@linux.vnet.ibm.com  wrote: On 09/07/2010 09:33 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 2:41

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:05 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:57 PM, Anthony Liguori aligu...@linux.vnet.ibm.com wrote: On 09/07/2010 09:49 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:34 PM, Kevin Wolfkw...@redhat.comwrote: Am 07.09.2010 15:41,

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:20 AM, Kevin Wolf wrote: Am 07.09.2010 17:11, schrieb Anthony Liguori: On 09/07/2010 10:02 AM, Kevin Wolf wrote: Am 07.09.2010 16:49, schrieb Anthony Liguori: Shouldn't it be a runtime option? You can use the very same image with copy-on-read or

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Kevin Wolf
Am 07.09.2010 17:30, schrieb Anthony Liguori: On 09/07/2010 10:20 AM, Kevin Wolf wrote: Am 07.09.2010 17:11, schrieb Anthony Liguori: Copy-on-read is, in many cases, a property of the backing file because it suggests that the backing file is either very slow or potentially volatile.

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:39 AM, Kevin Wolf wrote: Am 07.09.2010 17:30, schrieb Anthony Liguori: On 09/07/2010 10:20 AM, Kevin Wolf wrote: Am 07.09.2010 17:11, schrieb Anthony Liguori: Copy-on-read is, in many cases, a property of the backing file because it suggests that the backing

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Kevin Wolf
Am 07.09.2010 17:11, schrieb Anthony Liguori: On 09/07/2010 10:02 AM, Kevin Wolf wrote: Am 07.09.2010 16:49, schrieb Anthony Liguori: Shouldn't it be a runtime option? You can use the very same image with copy-on-read or copy-on-write and it will behave the same (execpt for performance),

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:09 AM, Stefan Hajnoczi wrote: Right, so that argues for an incremental interface like I started with :-) BTW, this whole discussion is also relevant for other background tasks like online defragmentation so keep that use-case in mind too. Right, I'm a little hesitant to