Re: [pve-devel] backup ceph high iops and slow
We read 64K blocks, so Don't for for backup, but drive-mirror have a granulary option to change the block size: # @granularity: #optional granularity of the dirty bitmap, default is 64K. # Must be a power of 2 between 512 and 64M. Although it would be much easier it read-ahead solve the problem ;-) Anybody have newest ceph with those read-ahead patches for testing? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] backup
Hi all, I just discovered that when Proxmox initiates the configured automatic backup it runs backups from each node in the cluster in parallel. I wonder whether this could be the cause for users now and then see some of the backups fails given the backup storage is to slow storage? Maybe a setting in the backup gui when checked prevents running parallel backups could solve this issue? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael at rasmussen dot cc http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xD3C9A00E mir at datanom dot net http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xE501F51C mir at miras dot org http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xE3E80917 -- /usr/games/fortune -es says: f u cn rd ths, u cn gt a gd jb n cmptr prgrmmng. pgpDfbJvZlFpI.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] backup
I just discovered that when Proxmox initiates the configured automatic backup it runs backups from each node in the cluster in parallel. I wonder whether this could be the cause for users now and then see some of the backups fails given the backup storage is to slow storage? Maybe a setting in the backup gui when checked prevents running parallel backups could solve this issue? It is quite easy to setup a separate job for each node. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] backup
On Sat, 18 Oct 2014 10:56:18 + Dietmar Maurer diet...@proxmox.com wrote: It is quite easy to setup a separate job for each node. I know, but maybe users are not aware of this complication. If it can remove a lot of noise on the forum from people with failing backups I think it is worth implementing. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael at rasmussen dot cc http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xD3C9A00E mir at datanom dot net http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xE501F51C mir at miras dot org http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xE3E80917 -- /usr/games/fortune -es says: I might have gone to West Point, but I was too proud to speak to a congressman. -- Will Rogers pgpeudVXyRP2C.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] backup
It is quite easy to setup a separate job for each node. I know, but maybe users are not aware of this complication. If it can remove a lot of noise on the forum from people with failing backups I think it is worth implementing. 1.) I don't think this is the reason for the problems. 2.) it work perfectly for 99% of people 3.) There is a workaround 4.) what you suggest is difficult to implement ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] backup
Well... I have just one node here, because the other server doesn't arrive yet, but I already create the cluster... And my backup task are extremaly slow! It a file about 23 GB, that take almost 5 hours. Ok! I ran backup over LVM-on-top-iSCSI VMIDNAMESTATUSTIMESIZEFILENAME 103Windows2012OK04:53:4122.52GB /BKPVMS/dump/vzdump-qemu-103-2014_10_17-18_30_02.vma.lzo TOTAL04:53:41 22.52GB On the other hand, this is another server , and a file with about 80 GB of size, take almost 45 minutes! VMIDNAMESTATUSTIMESIZEFILENAME 100Win2008OK00:44:5178.98GB /STORAGE03/dump/vzdump-qemu-100-2014_10_17-23_00_02.vma.lzo TOTAL00:44:51 78.98GB With sure something is wrong, but I can believe that paralel task has something to do with this, since I have just one node... Any advice will help? 2014-10-18 12:21 GMT-03:00 Dietmar Maurer diet...@proxmox.com: It is quite easy to setup a separate job for each node. I know, but maybe users are not aware of this complication. If it can remove a lot of noise on the forum from people with failing backups I think it is worth implementing. 1.) I don't think this is the reason for the problems. 2.) it work perfectly for 99% of people 3.) There is a workaround 4.) what you suggest is difficult to implement ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel -- -- Gilberto Ferreira ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] backup
On Sat, 18 Oct 2014 13:03:57 -0300 Gilberto Nunes gilberto.nune...@gmail.com wrote: Well... I have just one node here, because the other server doesn't arrive yet, but I already create the cluster... And my backup task are extremaly slow! It a file about 23 GB, that take almost 5 hours. Ok! I ran backup over LVM-on-top-iSCSI VMIDNAMESTATUSTIMESIZEFILENAME 103Windows2012OK04:53:4122.52GB /BKPVMS/dump/vzdump-qemu-103-2014_10_17-18_30_02.vma.lzo TOTAL04:53:41 22.52GB On the other hand, this is another server , and a file with about 80 GB of size, take almost 45 minutes! VMIDNAMESTATUSTIMESIZEFILENAME 100Win2008OK00:44:5178.98GB /STORAGE03/dump/vzdump-qemu-100-2014_10_17-23_00_02.vma.lzo TOTAL00:44:51 78.98GB This from my backup: 128ok 00:29:26 17.08GB /mnt/pve/qnap_nfs/dump/vzdump-qemu-128-2014_10_18-05_31_23.vma.lzo In generel I see backup speed around 25 MB/s on my relatively slow Qnap. One question: Do you make backup to the same storage as the VM's reside on? If this is the case then I would imagine that speed were halfed. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael at rasmussen dot cc http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xD3C9A00E mir at datanom dot net http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xE501F51C mir at miras dot org http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xE3E80917 -- /usr/games/fortune -es says: I keep seeing spots in front of my eyes. Did you ever see a doctor? No, just spots. pgpVk62XBlj56.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] backup
Oh Dear! Qnap! Pity of you! Qnap has only 128 MB of memory and low hardware... Here, I do backup locally, in an External HDD USB To one server, I saw here around 15-18 MB/s, but there's some ocasion that I saw less than 10 MB/s Compare to other server, I saw 60-80 MB/s... 2014-10-18 13:27 GMT-03:00 Michael Rasmussen m...@datanom.net: On Sat, 18 Oct 2014 13:03:57 -0300 Gilberto Nunes gilberto.nune...@gmail.com wrote: Well... I have just one node here, because the other server doesn't arrive yet, but I already create the cluster... And my backup task are extremaly slow! It a file about 23 GB, that take almost 5 hours. Ok! I ran backup over LVM-on-top-iSCSI VMIDNAMESTATUSTIMESIZEFILENAME 103Windows2012OK04:53:4122.52GB /BKPVMS/dump/vzdump-qemu-103-2014_10_17-18_30_02.vma.lzo TOTAL04:53:41 22.52GB On the other hand, this is another server , and a file with about 80 GB of size, take almost 45 minutes! VMIDNAMESTATUSTIMESIZEFILENAME 100Win2008OK00:44:5178.98GB /STORAGE03/dump/vzdump-qemu-100-2014_10_17-23_00_02.vma.lzo TOTAL00:44:51 78.98GB This from my backup: 128ok 00:29:26 17.08GB /mnt/pve/qnap_nfs/dump/vzdump-qemu-128-2014_10_18-05_31_23.vma.lzo In generel I see backup speed around 25 MB/s on my relatively slow Qnap. One question: Do you make backup to the same storage as the VM's reside on? If this is the case then I would imagine that speed were halfed. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael at rasmussen dot cc http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xD3C9A00E mir at datanom dot net http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xE501F51C mir at miras dot org http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xE3E80917 -- /usr/games/fortune -es says: I keep seeing spots in front of my eyes. Did you ever see a doctor? No, just spots. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel -- -- Gilberto Ferreira ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] backup
On Sat, 18 Oct 2014 10:56:18 AM Dietmar Maurer wrote: Maybe a setting in the backup gui when checked prevents running parallel backups could solve this issue? It is quite easy to setup a separate job for each node. Not if you need to exclude some vm's, but migrate them regularly for load balancing. -- Lindsay signature.asc Description: This is a digitally signed message part. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
