Re: [BackupPC-users] The usual questions looking for better ansers
On 04/29 10:18 , Richard Shaw wrote:
1. On the server, which is a Xeon Quad 2GHz machine, I've got a rsync
over ssh dump that's been running for over 12 hours which is about
300GB into a 800GB share. Perl seems to be the bottleneck on the
server and ssh is only using about 20-30% of one core on the client so
I'm assuming changing the ssh cypher will not help.
My experience is that changing the SSH cipher does sometimes help. I haven't
tested extensively; but it seems that sometimes it helps and sometimes it
doesn't make any difference.
I tend to use the following when backing up over slow links.
$Conf{RsyncClientCmd} = '$sshPath -C -o CompressionLevel=9 -c blowfish-cbc -q
-x -l rsyncbakup $host $rsyncPath $argList+';
(The '-l rsyncbakup' is because I have the clients set up with an SSH key
that does a sudo before executing the rsync command on the client side; so
no ssh-as-root needed).
Over a local link I'd remove the '-o CompressionLevel=9'; since with modern
local networks compression actually slows things down.
Other people have reported success with other ciphers. I don't remember
which one off the top of my head.
--
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com
--
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
[BackupPC-users] The usual questions looking for better ansers
I was recently sub-contracted to setup BackupPC for a business and everything's fine so far but I was hoping to make some improvements. 1. On the server, which is a Xeon Quad 2GHz machine, I've got a rsync over ssh dump that's been running for over 12 hours which is about 300GB into a 800GB share. Perl seems to be the bottleneck on the server and ssh is only using about 20-30% of one core on the client so I'm assuming changing the ssh cypher will not help. Is there any tips or tricks I can apply in this case? 2. Can more than one email address be added to the configuration for the EMailAdminUserName? 3. I have more than one share that needs to be backed up from the client but they want a different backup schedule for 1 of the shares. Am I going to have to setup fake/virtual hosts to accomplish this? The host-ip resolution is being handled by a hosts file so I could add aliases there so BackupPC would treat them as separate clients, right? -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] The usual questions looking for better ansers
On 4/29/2011 10:18 AM, Richard Shaw wrote: I was recently sub-contracted to setup BackupPC for a business and everything's fine so far but I was hoping to make some improvements. 1. On the server, which is a Xeon Quad 2GHz machine, I've got a rsync over ssh dump that's been running for over 12 hours which is about 300GB into a 800GB share. Perl seems to be the bottleneck on the server and ssh is only using about 20-30% of one core on the client so I'm assuming changing the ssh cypher will not help. Is there any tips or tricks I can apply in this case? Is this the 1st or 2nd full? It may improve by itself if you have the --checksum-seed=32761 option set so the server won't have to recompute the values. You could also look at the content and how it changes. Big files with small changes are bad, since the system has to reconstruct the copy with a mix of decompressing the old version and the changes from the network. Maybe there is something that you can exclude or handle some other way. Also, remember that incrementals without levels copy everything changed since the previous full and with levels have to do extra server-side work to merge the comparison view. 2. Can more than one email address be added to the configuration for the EMailAdminUserName? I think it should work to use a comma separated list, but an alternative would be an alias in the mail system. 3. I have more than one share that needs to be backed up from the client but they want a different backup schedule for 1 of the shares. Am I going to have to setup fake/virtual hosts to accomplish this? Yes. But make sure they understand that the pooling in backuppc means they wouldn't actually store more copies (unless they change) of files if they run all the shares at the more frequent schedule. The host-ip resolution is being handled by a hosts file so I could add aliases there so BackupPC would treat them as separate clients, right? Yes, but you could also use the ClientAlias setting in backuppc itself to make different host configurations point to the same real name or IP. -- Les Mikesell lesmikes...@gmail.com -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] The usual questions looking for better ansers
On Fri, Apr 29, 2011 at 10:54 AM, Les Mikesell lesmikes...@gmail.com wrote: On 4/29/2011 10:18 AM, Richard Shaw wrote: I was recently sub-contracted to setup BackupPC for a business and everything's fine so far but I was hoping to make some improvements. 1. On the server, which is a Xeon Quad 2GHz machine, I've got a rsync over ssh dump that's been running for over 12 hours which is about 300GB into a 800GB share. Perl seems to be the bottleneck on the server and ssh is only using about 20-30% of one core on the client so I'm assuming changing the ssh cypher will not help. Is there any tips or tricks I can apply in this case? Is this the 1st or 2nd full? It may improve by itself if you have the --checksum-seed=32761 option set so the server won't have to recompute the values. You could also look at the content and how it changes. Big files with small changes are bad, since the system has to reconstruct the copy with a mix of decompressing the old version and the changes from the network. Maybe there is something that you can exclude or handle some other way. Also, remember that incrementals without levels copy everything changed since the previous full and with levels have to do extra server-side work to merge the comparison view. It's the 1st full so I guess that wouldn't help... Is that a safe option to keep using all the time? The files are commercial animation files so I'm guessing they are large. I was looking for a utility like 'du' except for file size distribution but didn't really find anything. They can't really be excluded since they are the point of the backup... 2. Can more than one email address be added to the configuration for the EMailAdminUserName? I think it should work to use a comma separated list, but an alternative would be an alias in the mail system. I'll give it a try and see if it works on my test server (virtualbox) and report back. 3. I have more than one share that needs to be backed up from the client but they want a different backup schedule for 1 of the shares. Am I going to have to setup fake/virtual hosts to accomplish this? Yes. But make sure they understand that the pooling in backuppc means they wouldn't actually store more copies (unless they change) of files if they run all the shares at the more frequent schedule. That shouldn't be a problem, these are different directories with dissimilar files, so not a pooling opportunity. The host-ip resolution is being handled by a hosts file so I could add aliases there so BackupPC would treat them as separate clients, right? Yes, but you could also use the ClientAlias setting in backuppc itself to make different host configurations point to the same real name or IP. Can you point me to some documentation on how to use ClientAlias? Surprisingly googling backuppc clientalias didn't seem to get me what I needed. And I found no instances of that in the basic documentation. Thanks, Richard -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] The usual questions looking for better ansers
On Fri, Apr 29, 2011 at 01:09:24PM -0500, Richard Shaw wrote: On Fri, Apr 29, 2011 at 10:54 AM, Les Mikesell lesmikes...@gmail.com wrote: On 4/29/2011 10:18 AM, Richard Shaw wrote: The host-ip resolution is being handled by a hosts file so I could add aliases there so BackupPC would treat them as separate clients, right? Yes, but you could also use the ClientAlias setting in backuppc itself to make different host configurations point to the same real name or IP. Can you point me to some documentation on how to use ClientAlias? Surprisingly googling backuppc clientalias didn't seem to get me what I needed. And I found no instances of that in the basic documentation. Les almost got it right it's ClientNameAlias http://backuppc.sourceforge.net/faq/BackupPC.html#_conf_clientnamealias_ Enjoy. -- -- rouilj John Rouillard System Administrator Renesys Corporation 603-244-9084 (cell) 603-643-9300 x 111 -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] The usual questions looking for better ansers
On Fri, Apr 29, 2011 at 1:24 PM, John Rouillard rouilj-backu...@renesys.com wrote: Les almost got it right it's ClientNameAlias http://backuppc.sourceforge.net/faq/BackupPC.html#_conf_clientnamealias_ Thanks! I think I understand how that works! Richard -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] The usual questions looking for better ansers
On 4/29/2011 1:09 PM, Richard Shaw wrote:
Is this the 1st or 2nd full? It may improve by itself if you have the
--checksum-seed=32761 option set so the server won't have to recompute
the values.
It's the 1st full so I guess that wouldn't help... Is that a safe
option to keep using all the time?
Yes - with it on, the block checksums needed to verify the file are
saved on the 2nd run so the server doesn't have to uncompress and
recompute them on subsequent full runs.
The files are commercial animation files so I'm guessing they are
large. I was looking for a utility like 'du' except for file size
distribution but didn't really find anything. They can't really be
excluded since they are the point of the backup...
If they don't change, things will go faster later. Incrementals will
skip over on the directory timestamp/length match. Fulls will do a
block checksum verify.
3. I have more than one share that needs to be backed up from the
client but they want a different backup schedule for 1 of the shares.
Am I going to have to setup fake/virtual hosts to accomplish this?
Yes. But make sure they understand that the pooling in backuppc means
they wouldn't actually store more copies (unless they change) of files
if they run all the shares at the more frequent schedule.
That shouldn't be a problem, these are different directories with
dissimilar files, so not a pooling opportunity.
What I mean is that multiple runs of the same share are pooled, so while
there may be other (load, network traffic) reasons to back up some parts
less frequently, doing all the shares at the most frequent desired
schedule probably won't take a lot more server space.
The host-ip resolution is being handled by a hosts file so I could
add aliases there so BackupPC would treat them as separate clients,
right?
Yes, but you could also use the ClientAlias setting in backuppc itself
to make different host configurations point to the same real name or IP.
Can you point me to some documentation on how to use ClientAlias?
Surprisingly googling backuppc clientalias didn't seem to get me
what I needed. And I found no instances of that in the basic
documentation.
Sorry, it is actually $Conf{ClientNameAlias}. You can use dummy
hostnames so you can control the schedule separately but override the
actual target with this setting.
--
Les Mikesell
lesmikes...@gmail.com
--
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] The usual questions looking for better ansers
On Fri, Apr 29, 2011 at 1:42 PM, Les Mikesell lesmikes...@gmail.com wrote: On 4/29/2011 1:09 PM, Richard Shaw wrote: 3. I have more than one share that needs to be backed up from the client but they want a different backup schedule for 1 of the shares. Am I going to have to setup fake/virtual hosts to accomplish this? Yes. But make sure they understand that the pooling in backuppc means they wouldn't actually store more copies (unless they change) of files if they run all the shares at the more frequent schedule. That shouldn't be a problem, these are different directories with dissimilar files, so not a pooling opportunity. What I mean is that multiple runs of the same share are pooled, so while there may be other (load, network traffic) reasons to back up some parts less frequently, doing all the shares at the most frequent desired schedule probably won't take a lot more server space. Yeah, the main backup is their projects folders, the rest we were going to backup less frequently mainly due to their lesser importance more than any other reason, but also due to load and traffic as this is the master for their rendering farm. Thanks, Richard -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
