Re: [pve-devel] snapshot improvements

2012-09-29 Thread Stefan Priebe - Profihost AG
Am 29.09.2012 um 08:26 schrieb Dietmar Maurer diet...@proxmox.com: I thought of something like: ssh cloud1-1200.de-nserver.de zfs send JBOD01Pool/vm-105-disk- 1@testsnap | zfs recv tank/abc or ssh cloud1-1200.de-nserver.de zfs send JBOD01Pool/vm-105-disk-1@spriebetest | gzip

Re: [pve-devel] snapshot improvements

2012-09-29 Thread Dietmar Maurer
Are your aware of the fact that ssh needs to encrypt/decrypt all data. This needs much CPU power and is slow (still no AES support in libs). Oh we could also use netcat instead. How exactly? ___ pve-devel mailing list pve-devel@pve.proxmox.com

Re: [pve-devel] snapshot improvements

2012-09-28 Thread Alexandre DERUMIER
. - Mail original - De: Dietmar Maurer diet...@proxmox.com À: Alexandre DERUMIER aderum...@odiso.com Cc: pve-devel@pve.proxmox.com, Stefan Priebe s.pri...@profihost.ag Envoyé: Vendredi 28 Septembre 2012 06:40:10 Objet: RE: [pve-devel] snapshot improvements Subject: Re: [pve-devel

Re: [pve-devel] snapshot improvements

2012-09-28 Thread Dietmar Maurer
what do you mean by snapshot used by another snapshot? ex: image-snap1-snap2-you are here you can delete snap1 without any problem, and without need to merge. Really? - last time I tried I got 'snapshot in use' (or something like that). ___

Re: [pve-devel] snapshot improvements

2012-09-28 Thread Alexandre DERUMIER
maybe do you have cloned it ? - Mail original - De: Dietmar Maurer diet...@proxmox.com À: Alexandre DERUMIER aderum...@odiso.com Cc: pve-devel@pve.proxmox.com, Stefan Priebe s.pri...@profihost.ag Envoyé: Vendredi 28 Septembre 2012 11:32:01 Objet: RE: [pve-devel] snapshot

Re: [pve-devel] snapshot improvements

2012-09-28 Thread Dietmar Maurer
Objet: RE: [pve-devel] snapshot improvements what do you mean by snapshot used by another snapshot? ex: image-snap1-snap2-you are here you can delete snap1 without any problem, and without need to merge. Really? - last time I tried I got 'snapshot in use' (or something like

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Stefan Priebe - Profihost AG
Am 27.09.2012 06:43, schrieb Dietmar Maurer: I also tried todo backups with snapshots but this doesn't seem to work: INFO: starting new backup job: vzdump 100 --remove 0 --mode snapshot -- compress lzo --storage backuplocal --node serv121 INFO: Starting Backup of VM 100 (qemu) INFO: status =

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
I also don't think that all storage can export datas with snapshots inside. rbd can export an image from a snapshot, nexenta and sheepdog too. How can you access snapshot data with nexenta? ___ pve-devel mailing list pve-devel@pve.proxmox.com

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
The idea is that we do not backup any snapshot data. The vzdump would only include the data of from the running instance. I guess that is OK? But isn't the correct way to make a snapshot and then compress the snapshot? How can vzdump verify that the data integrity is fine? Sorry, I do not

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Alexandre DERUMIER
Cc: pve-devel@pve.proxmox.com, Stefan Priebe s.pri...@profihost.ag Envoyé: Jeudi 27 Septembre 2012 09:04:31 Objet: RE: [pve-devel] snapshot improvements I also don't think that all storage can export datas with snapshots inside. rbd can export an image from a snapshot, nexenta and sheepdog

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
I think I have already respond to that ;) 2 ways : - clone the snapshot and export image through iscsi and backup it - use zfs send through ssh (zfs send image1@snap1 /imagefile) Both ways are clumsy. Also, nexenta snapshot support is quite unusable, because you can't delete

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
The idea is that we do not backup any snapshot data. The vzdump would only include the data of from the running instance. I guess that is OK? But isn't the correct way to make a snapshot and then compress the snapshot? How can vzdump verify that the data integrity is fine? I guess I found

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Stefan Priebe - Profihost AG
Am 27.09.2012 09:26, schrieb Dietmar Maurer: The idea is that we do not backup any snapshot data. The vzdump would only include the data of from the running instance. I guess that is OK? But isn't the correct way to make a snapshot and then compress the snapshot? How can vzdump verify that the

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
ah OK but wouldn't it be nice to be able to backup live snapshots? compress and store them on a seperate NFS server? Also Proxmox only allows to schedule beckups not to schedule snapshots ;-) Sure, that would be nice. I already have very detailed plans how to do that (but no budget).

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
And if one would consider to backup snapshots, I am quite sure he only wants to backup shared data once. A simple data export would duplicate large amounts of data. Subject: Re: [pve-devel] snapshot improvements I think I have already respond to that ;) 2 ways : - clone the snapshot

Re: [pve-devel] snapshot improvements

2012-09-27 Thread Dietmar Maurer
Subject: Re: [pve-devel] snapshot improvements do you mean : backup the main image then backup each snasphot increment ? Yes, something like that. zfs send can do incremental backup with zfs send -I http://docs.oracle.com/cd/E19082-01/817-2271/gfwqb/index.html Ah, good to know

Re: [pve-devel] snapshot improvements

2012-09-26 Thread Stefan Priebe
Am 24.09.2012 10:50, schrieb Dietmar Maurer: I tried to improve snapshot behavior a bit: - take snapshot is now a background task (does not block monitor) - try to save VM state while VM is online So downtime during snapshot should be shorter now, especially if the VM use much RAM. Feel free

Re: [pve-devel] snapshot improvements

2012-09-26 Thread Dietmar Maurer
I also tried todo backups with snapshots but this doesn't seem to work: INFO: starting new backup job: vzdump 100 --remove 0 --mode snapshot -- compress lzo --storage backuplocal --node serv121 INFO: Starting Backup of VM 100 (qemu) INFO: status = running ERROR: Backup of VM 100 failed -

Re: [pve-devel] snapshot improvements

2012-09-25 Thread Dietmar Maurer
do you think image = size of memory if enough ? (if we have some bytes more with incremental ?) PVE/QemuServer.pm: my $driver_state_size = 32; # assume 32MB is enough to safe all driver state; my $size = $conf-{memory} + $driver_state_size; Unfortunately, the qemu code seem to

Re: [pve-devel] snapshot improvements

2012-09-25 Thread Alexandre DERUMIER
aderum...@odiso.com Cc: pve-devel@pve.proxmox.com Envoyé: Mardi 25 Septembre 2012 10:37:42 Objet: RE: [pve-devel] snapshot improvements do you think image = size of memory if enough ? (if we have some bytes more with incremental ?) PVE/QemuServer.pm: my $driver_state_size = 32

Re: [pve-devel] snapshot improvements

2012-09-24 Thread Dietmar Maurer
How is it possible to save the vmstate if the vm is not paused ? (with a lot a memory write access by example ?) I currently allocate am image equal to the size of the VM memory. Then I simply do an incremental state save (like a vm migration), and keep the VM running until (saved_bytes +

Re: [pve-devel] snapshot improvements

2012-09-24 Thread Alexandre DERUMIER
: Lundi 24 Septembre 2012 11:52:04 Objet: RE: [pve-devel] snapshot improvements How is it possible to save the vmstate if the vm is not paused ? (with a lot a memory write access by example ?) I currently allocate am image equal to the size of the VM memory. Then I simply do an incremental

Re: [pve-devel] snapshot improvements

2012-09-24 Thread Dietmar Maurer
do you think image = size of memory if enough ? (if we have some bytes more with incremental ?) PVE/QemuServer.pm: my $driver_state_size = 32; # assume 32MB is enough to safe all driver state; my $size = $conf-{memory} + $driver_state_size; Also, any problem if some datas in