Re: [BackupPC-users] Second Destination
On 2016-11-21 11:01, Les Mikesell wrote: > On Mon, Nov 21, 2016 at 6:29 AM, Bob of Donelson Trophy > wrote: > >> I backed up a small Active Directory Domain Controller (Samba4). It is about >> 5Gb in size and (full) backed up in about an hour. Slower than the local >> Backuppc but, not a lot slower. > > Note that samba xfers are going to transfer everything on every full run. > >> Then I started the backup of one of the ADDC member file servers . . . about >> 250Gb (I think) . . . Backuppc (thru vpn) has been running now for about 21 >> hours for this first, full backup. This backup took about five or six hours >> locally. > > Is this also samba or are you using rsync for this? > >> I'm guessing it will finish soon . . . I hope. >> >> This idea of doing the initial backup (of the second machine) on the local >> lan and then moving it to the second location, I do not see this as very >> practical. Every week (6.97 days) Backuppc does a full backup, does this >> mean I need to move the machine every 6.97 days to the local lan? > > With the rsync or rysyncd xfer methods, even on full runs only the > changes are sent over the wire although the client reads all the files > to do a block checksum comparison so it may still take a long time to > complete even when not using a lot of bandwidth. > >> Surely there is some compression options that could be put in place to >> better stream the data thru the vpn and improve the backup speed? >> >> So I add "-C" to the "RsyncClientCmd sshpath" or "RsyncArgs"? >> >> Aren't the "-q -x -l" default options in the "RsyncClientCmd sshpath" ssh >> options? (Therefore the "-C" addition goes with them?) > > Yes, it goes in the sshpath - or you can also control compression with > the sshd configuration on the client side. However, it is only used > with the rsync xfer method. Rsyncd and samba do not use the ssh > layer. > > -- > Les Mikesell > lesmikes...@gmail.com All my Samba machines (DC's and member servers) are Debian/Ubuntu based and being backed up via rsync. (Never had a reason to try rsyncd.) My W7/W10 clients are all workstations with folder redirection so I do not use Backuppc to back them up. (Just clonezilla images once a week for them.) Thanks to Adam Goryachev response from earlier, you have reiterated the reason to do initial rsync backup on the local lan and then relocate the machine. (I had read this in the past and forgotten that fact.) That is easy to do, for me. But, I am going to try adding the "-C" to the client sshpath just to test and see what happens. Thank for your help on this. I really appreciate it. -- ___ Bob Wooden of Donelson Trophy-- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Second Destination
On Mon, Nov 21, 2016 at 6:29 AM, Bob of Donelson Trophy wrote: > > I backed up a small Active Directory Domain Controller (Samba4). It is about > 5Gb in size and (full) backed up in about an hour. Slower than the local > Backuppc but, not a lot slower. > Note that samba xfers are going to transfer everything on every full run. > Then I started the backup of one of the ADDC member file servers . . . about > 250Gb (I think) . . . Backuppc (thru vpn) has been running now for about 21 > hours for this first, full backup. This backup took about five or six hours > locally. Is this also samba or are you using rsync for this? > I'm guessing it will finish soon . . . I hope. > > This idea of doing the initial backup (of the second machine) on the local > lan and then moving it to the second location, I do not see this as very > practical. Every week (6.97 days) Backuppc does a full backup, does this > mean I need to move the machine every 6.97 days to the local lan? With the rsync or rysyncd xfer methods, even on full runs only the changes are sent over the wire although the client reads all the files to do a block checksum comparison so it may still take a long time to complete even when not using a lot of bandwidth. > Surely there is some compression options that could be put in place to > better stream the data thru the vpn and improve the backup speed? > > So I add "-C" to the "RsyncClientCmd sshpath" or "RsyncArgs"? > > Aren't the "-q -x -l" default options in the "RsyncClientCmd sshpath" ssh > options? (Therefore the "-C" addition goes with them?) Yes, it goes in the sshpath - or you can also control compression with the sshd configuration on the client side. However, it is only used with the rsync xfer method. Rsyncd and samba do not use the ssh layer. -- Les Mikesell lesmikes...@gmail.com -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Second Destination
On 21/11/16 23:29, Bob of Donelson Trophy wrote: On 2016-11-20 17:58, Les Mikesell wrote: On Sun, Nov 20, 2016 at 4:32 PM, Bob of Donelson Trophy mailto:b...@donelsontrophy.net>> wrote: I am currently testing backing up one client machine from the other end of the vpn. I used the client machines ipaddress instead of it's hostname. The ssh key was copied and the backup began as expected. So, the different ip subnet did not matter to the backuppc machine. (First time I had ever tried backing up thru the vpn.) I also expected the backup to be slower thru the vpn than the local backuppc machine there and it appears to be that, slower. I figured, for now, keep it simple and just try this and see what happens. When the backup completes I will compare the two backuppc machine client data. I'm looking for redundancy with two backups per client machine in two different geographical locations, one local lan and the other at my house (other end of vpn.) Backup has been running about three hours now. Like you said, the two subnets are well translated by the vpn link. As far as his "string", I'm still studying what the "options" mean and might his way be a faster backup method or not. These are only thoughts at this point. The -C enables compression for the ssh and is likely to help considerably - unless your VPN is also doing compression. The initial copy of each client is likely to take a long time, depending on the speed of your network connections, but subsequent runs with rsync will be much faster, only copying the differences. If the initial full runs are impractical, it might work to initial set the 2nd server up locally, then move it to the offsite location with the initial data in place - or ship a drive for a similar machine. (First my apologizes, my webmail client has a glitch. Not a story for this mailing list but my reply will look like an extension of the previous message and can cause confusion during reading . . . you'll see what I mean if the "glitch" happens.) Thanks for the info, Les. I backed up a small Active Directory Domain Controller (Samba4). It is about 5Gb in size and (full) backed up in about an hour. Slower than the local Backuppc but, not a lot slower. Then I started the backup of one of the ADDC member file servers . . . about 250Gb (I think) . . . Backuppc (thru vpn) has been running now for about 21 hours for this first, full backup. This backup took about five or six hours locally. I'm guessing it will finish soon . . . I hope. This idea of doing the initial backup (of the second machine) on the local lan and then moving it to the second location, I do not see this as very practical. Every week (6.97 days) Backuppc does a full backup, does this mean I need to move the machine every 6.97 days to the local lan? Surely there is some compression options that could be put in place to better stream the data thru the vpn and improve the backup speed? So I add "-C" to the "RsyncClientCmd sshpath" or "RsyncArgs"? Aren't the "-q -x -l" default options in the "RsyncClientCmd sshpath" ssh options? (Therefore the "-C" addition goes with them? I think the point you are missing is that backuppc using rsync for a full backup won't actually transfer all of the content of all the files. This will only happen on the first backup. Future backups will only transfer the portion of the files which are modified since the last incremental (or full, depending on which is more recent), and the content of new files. You should use SSH compression if your VPN doesn't do compression by itself, or you could experiment to see which combination of compression works best for your system (ssh compression + vpn compression). Generally, I'd simply enable compression on vpn, since I would have that on for other clients/needs anyway, and then trust that rsync will already be reducing the amount of data that needs to be transferred. I hope that help clear up why you only need the backuppc server on the same lan for the initial backup (or just be really patient for the first remote backup). If you want a very rough estimate on the time it will take, look at the disk space in use on the backup target (150GB I think you said), and then calculate the length of time to transfer that amount of data to across your link, then add another 25%. Regards, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Second Destination
On 2016-11-20 17:58, Les Mikesell wrote: > On Sun, Nov 20, 2016 at 4:32 PM, Bob of Donelson Trophy > wrote: > >> I am currently testing backing up one client machine from the other end of >> the vpn. I used the client machines ipaddress instead of it's hostname. The >> ssh key was copied and the backup began as expected. So, the different ip >> subnet did not matter to the backuppc machine. (First time I had ever tried >> backing up thru the vpn.) I also expected the backup to be slower thru the >> vpn than the local backuppc machine there and it appears to be that, slower. >> >> I figured, for now, keep it simple and just try this and see what happens. >> >> When the backup completes I will compare the two backuppc machine client >> data. I'm looking for redundancy with two backups per client machine in two >> different geographical locations, one local lan and the other at my house >> (other end of vpn.) Backup has been running about three hours now. >> >> Like you said, the two subnets are well translated by the vpn link. >> >> As far as his "string", I'm still studying what the "options" mean and might >> his way be a faster backup method or not. These are only thoughts at this >> point. > > The -C enables compression for the ssh and is likely to help > considerably - unless your VPN is also doing compression. The > initial copy of each client is likely to take a long time, depending > on the speed of your network connections, but subsequent runs with > rsync will be much faster, only copying the differences. If the > initial full runs are impractical, it might work to initial set the > 2nd server up locally, then move it to the offsite location with the > initial data in place - or ship a drive for a similar machine. (First my apologizes, my webmail client has a glitch. Not a story for this mailing list but my reply will look like an extension of the previous message and can cause confusion during reading . . . you'll see what I mean if the "glitch" happens.) Thanks for the info, Les. I backed up a small Active Directory Domain Controller (Samba4). It is about 5Gb in size and (full) backed up in about an hour. Slower than the local Backuppc but, not a lot slower. Then I started the backup of one of the ADDC member file servers . . . about 250Gb (I think) . . . Backuppc (thru vpn) has been running now for about 21 hours for this first, full backup. This backup took about five or six hours locally. I'm guessing it will finish soon . . . I hope. This idea of doing the initial backup (of the second machine) on the local lan and then moving it to the second location, I do not see this as very practical. Every week (6.97 days) Backuppc does a full backup, does this mean I need to move the machine every 6.97 days to the local lan? Surely there is some compression options that could be put in place to better stream the data thru the vpn and improve the backup speed? So I add "-C" to the "RsyncClientCmd sshpath" or "RsyncArgs"? Aren't the "-q -x -l" default options in the "RsyncClientCmd sshpath" ssh options? (Therefore the "-C" addition goes with them?) -- ___ Bob Wooden of Donelson Trophy-- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
