Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-03-06 Thread Dietmar Maurer
> > > > Or can we simply add a header to each write with additional information? > > > > > > Let's discuss it once we have a clearer picture of what that > > > additional info would be. > > > > OK, just discovered such case. > > > > For example it would be nice to pass the size of the devices someh

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-03-06 Thread Stefan Hajnoczi
On Mon, Feb 25, 2013 at 04:35:09PM +, Dietmar Maurer wrote: > > >>> If we use nbd, how can we pass additional information to the other > > >>> side, for example information about unallocated regions? > > >>> > > >> > > >> You can either send trim commands, or just skip those regions and let > >

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-25 Thread Dietmar Maurer
> >>> If we use nbd, how can we pass additional information to the other > >>> side, for example information about unallocated regions? > >>> > >> > >> You can either send trim commands, or just skip those regions and let > >> the other side figure it out. > > > > But we are lost if we want to tran

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-25 Thread Stefan Hajnoczi
On Sat, Feb 23, 2013 at 09:05:37AM +, Dietmar Maurer wrote: > > > Sure. nbd+unix:///exportname?socket=path is the new URI syntax, I > > > honestly forgot the old one. SCM_CREDENTIALS checks (qemu-nbd --pid > > > or something like that) is not supported, but patches would be very > > > welcom

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-25 Thread Paolo Bonzini
Il 25/02/2013 13:59, Dietmar Maurer ha scritto: unix sockets works with qemu nbd code? >>> >>> Sure. nbd+unix:///exportname?socket=path is the new URI syntax, I >>> honestly forgot the old one. SCM_CREDENTIALS checks (qemu-nbd --pid >>> or something like that) is not supported, but patches w

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-25 Thread Dietmar Maurer
> >> unix sockets works with qemu nbd code? > > > > Sure. nbd+unix:///exportname?socket=path is the new URI syntax, I > > honestly forgot the old one. SCM_CREDENTIALS checks (qemu-nbd --pid > > or something like that) is not supported, but patches would be very welcome. > > Yes, this is better t

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-25 Thread Dietmar Maurer
> > So if I want to use the code from nbd.c, I need to write a specialized > > BlockDriver for the vma format (to pass that to nbd_export_new())? > > Yes. But I believe that would be a good thing to do anyway. For one thing, > it > gives you "automatic" coverage via qemu-iotests. But the only

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-24 Thread Paolo Bonzini
> > > Sure. nbd+unix:///exportname?socket=path is the new URI syntax, I > > > honestly forgot the old one. SCM_CREDENTIALS checks (qemu-nbd --pid > > > or something like that) is not supported, but patches would be > > > very welcome. > > > > Yes, this is better than my tcp suggestion. > > So i

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-24 Thread Dietmar Maurer
> -Original Message- > From: Paolo Bonzini [mailto:pbonz...@redhat.com] > Sent: Freitag, 22. Februar 2013 19:07 > To: Dietmar Maurer > Cc: Stefan Hajnoczi; Kevin Wolf; qemu-devel@nongnu.org; Wenchao Xia > Subject: Re: [PATCH v4 0/6] Efficient VM backup for qemu > > Il 22/02/2013 18:57, D

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-23 Thread Dietmar Maurer
> > Sure. nbd+unix:///exportname?socket=path is the new URI syntax, I > > honestly forgot the old one. SCM_CREDENTIALS checks (qemu-nbd --pid > > or something like that) is not supported, but patches would be very welcome. > > Yes, this is better than my tcp suggestion. So if I want to use the

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Paolo Bonzini
Il 22/02/2013 18:57, Dietmar Maurer ha scritto: >>> If we use nbd, how can we pass additional information to the other >>> side, for example information about unallocated regions? >>> >> >> You can either send trim commands, or just skip those regions and let the >> other >> side figure it out. >

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Dietmar Maurer
> > If we use nbd, how can we pass additional information to the other > > side, for example information about unallocated regions? > > > > You can either send trim commands, or just skip those regions and let the > other > side figure it out. But we are lost if we want to transfer something els

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Paolo Bonzini
Il 22/02/2013 18:49, Dietmar Maurer ha scritto: > >> unix sockets works with qemu nbd code? >>> > > >>> > > Sure. nbd+unix:///exportname?socket=path is the new URI syntax, I >>> > > honestly forgot the old one. SCM_CREDENTIALS checks (qemu-nbd --pid >>> > > or something like that) is not sup

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Dietmar Maurer
> On Fri, Feb 22, 2013 at 2:07 PM, Paolo Bonzini wrote: > > Il 22/02/2013 14:02, Dietmar Maurer ha scritto: > >>> But you can always sandbox using SELinux, if you care about that, or > >>> use a Unix socket + SCM_CREDENTIALS. > >> > >> unix sockets works with qemu nbd code? > > > > Sure. nbd+uni

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Stefan Hajnoczi
On Fri, Feb 22, 2013 at 2:07 PM, Paolo Bonzini wrote: > Il 22/02/2013 14:02, Dietmar Maurer ha scritto: >>> But you can always sandbox using SELinux, if you care about that, or use a >>> Unix >>> socket + SCM_CREDENTIALS. >> >> unix sockets works with qemu nbd code? > > Sure. nbd+unix:///exportn

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Stefan Hajnoczi
On Fri, Feb 22, 2013 at 11:39 AM, Kevin Wolf wrote: > Am 22.02.2013 um 11:31 hat Stefan Hajnoczi geschrieben: >> On Thu, Feb 21, 2013 at 03:56:15PM +, Dietmar Maurer wrote: >> > > On Wed, Feb 20, 2013 at 04:04:49PM +, Dietmar Maurer wrote: >> I added the 'wait-for' attribute which tells QM

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Paolo Bonzini
Il 22/02/2013 14:02, Dietmar Maurer ha scritto: > > Why is this needed? >>> Security? I don't want that another process can write nonsense into my >> backup. >> >> They can already write nonsense to your iSCSI target, can't they? > > I am more concerned about software bugs. You need to find a

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Dietmar Maurer
> >> > Why is this needed? > > Security? I don't want that another process can write nonsense into my > backup. > > They can already write nonsense to your iSCSI target, can't they? I am more concerned about software bugs. You need to find a free port, and then pass that port to kvm. If the ori

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Paolo Bonzini
Il 22/02/2013 12:51, Dietmar Maurer ha scritto: >> You can also use the named-export support. Something like >> > >> >--nbd=127.0.0.1:1235 >> >--blockdev=virtio-drive0 --blockdev=virtio-drive1 --blockdev=ide0-hd0 >> > >> > etc. >> > >> > Then QEMU can migrate to nbd://127.0.0.1:1235/vir

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Dietmar Maurer
> You can also use the named-export support. Something like > >--nbd=127.0.0.1:1235 >--blockdev=virtio-drive0 --blockdev=virtio-drive1 --blockdev=ide0-hd0 > > etc. > > Then QEMU can migrate to nbd://127.0.0.1:1235/virtio-drive0. > > > Or how can we guarantee that only one kvm process c

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Paolo Bonzini
Il 22/02/2013 12:13, Dietmar Maurer ha scritto: > Opening several NBD sockets 'nbd:127.0.0.1:1235' seems dangerous to me. One > backup task > and easily write to the wrong port - there is no protection? You can also use the named-export support. Something like --nbd=127.0.0.1:1235 --block

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Dietmar Maurer
Opening several NBD sockets 'nbd:127.0.0.1:1235' seems dangerous to me. One backup task and easily write to the wrong port - there is no protection? Or how can we guarantee that only one kvm process can write to that NBD device? > transaction actions=[{'type': 'migrate-async', >

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Kevin Wolf
Am 22.02.2013 um 11:31 hat Stefan Hajnoczi geschrieben: > On Thu, Feb 21, 2013 at 03:56:15PM +, Dietmar Maurer wrote: > > > On Wed, Feb 20, 2013 at 04:04:49PM +, Dietmar Maurer wrote: > > > > > The backup writer abstraction is a special case interface somewhere > > > > > between an image fo

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-22 Thread Stefan Hajnoczi
On Thu, Feb 21, 2013 at 03:56:15PM +, Dietmar Maurer wrote: > > On Wed, Feb 20, 2013 at 04:04:49PM +, Dietmar Maurer wrote: > > > > The backup writer abstraction is a special case interface somewhere > > > > between an image format and live migration. I'd prefer it if the > > > > backup bl

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-21 Thread Paolo Bonzini
Il 21/02/2013 16:56, Dietmar Maurer ha scritto: >> > In _your_ use case. That means that we should support using non-seekable >> > pipes, but it doesn't mean that any other use case is irrelevant. In fact >> > I think >> > backing up to a normal raw or qcow2 image file is the interesting case for

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-21 Thread Dietmar Maurer
> On Wed, Feb 20, 2013 at 04:04:49PM +, Dietmar Maurer wrote: > > > The backup writer abstraction is a special case interface somewhere > > > between an image format and live migration. I'd prefer it if the > > > backup block job stuck to BlockDriverState as the destination for backup > data.

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-21 Thread Kevin Wolf
On Wed, Feb 20, 2013 at 04:04:49PM +, Dietmar Maurer wrote: > > The backup writer abstraction is a special case interface somewhere > > between an image format and live migration. I'd prefer it if the backup > > block job stuck to BlockDriverState as the destination for backup data. > > Then y

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-21 Thread Kevin Wolf
On Thu, Feb 21, 2013 at 06:53:24AM +, Dietmar Maurer wrote: > > How would VMA fit in if the backup block job took a BlockDriverState > > destination and vmsave worked atomically with backup block jobs? You could > > have QEMU write to NBD server ports that a VMA writer process has open. > > VM

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-21 Thread Stefan Hajnoczi
On Wed, Feb 20, 2013 at 04:04:49PM +, Dietmar Maurer wrote: > > Interesting series, the backup block job makes sense to me. Regarding > > efficiency, I think incremental backup is a must, otherwise regular backups > > using this approach won't really be a win over backing files. > > Increment

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-21 Thread Stefan Hajnoczi
On Thu, Feb 21, 2013 at 06:53:24AM +, Dietmar Maurer wrote: > > Interesting series, the backup block job makes sense to me. Regarding > > efficiency, I think incremental backup is a must, > > One can easily implement incremental backup on top of this patches. That is > why > I introduced th

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-20 Thread Dietmar Maurer
> Interesting series, the backup block job makes sense to me. Regarding > efficiency, I think incremental backup is a must, One can easily implement incremental backup on top of this patches. That is why I introduced the BackupDriver abstraction. > otherwise regular backups using > this approac

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-20 Thread Dietmar Maurer
> Interesting series, the backup block job makes sense to me. Regarding > efficiency, I think incremental backup is a must, otherwise regular backups > using this approach won't really be a win over backing files. Incremental backup is something different, not touched by this code. You can add th

Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-20 Thread Stefan Hajnoczi
On Wed, Feb 20, 2013 at 10:31:57AM +0100, Dietmar Maurer wrote: > This series provides a way to efficiently backup VMs. > > * Backup to a single archive file > * Backup contain all data to restore VM (full backup) > * Do not depend on storage type or image format > * Avoid use of temporary storage

[Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

2013-02-20 Thread Dietmar Maurer
This series provides a way to efficiently backup VMs. * Backup to a single archive file * Backup contain all data to restore VM (full backup) * Do not depend on storage type or image format * Avoid use of temporary storage * store sparse images efficiently The file docs/backup.txt contains more d