Re: [BackupPC-users] Backuppc is Hammering My Clients
On Wed, 2012-02-01 at 20:24 -0500, Jeffrey J. Kosowsky wrote: Les Mikesell wrote at about 07:40:20 -0600 on Wednesday, February 1, 2012: On Wed, Feb 1, 2012 at 2:30 AM, Kimball Larsen quang...@gmail.com wrote: Do any have local time machine backups that might be included? No, time machine is on external drives, specifically excluded from backups. It might be worth checking that the excludes work and the links that make it show on the desktop aren't being followed. Or directories with very large numbers of files? This I can check on. What is considered very large numbers of files? More than 1024? More than 102400? It would be relative to the amount of RAM available - probably millions. I have no trouble backing up half a million files on a system with just 512MB. Surely, if you have high-powered relatively new PCs you will have many 4+ Gigabytes so it is unlikely that RAM swapping will be the problem. Plus any 3.0 version of rsync uses RAM quite efficiently. I find that the speed of the underlying storage is more of a factor than the ram or cpu limits -- Tim Fletcher t...@night-shade.org.uk -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
On Jan 31, 2012, at 5:10 PM, Les Mikesell wrote: On Tue, Jan 31, 2012 at 5:25 PM, Kimball Larsen quang...@gmail.com wrote: We are a small office (6 employees) with a mixture of windows and mac machines sitting on desks. I have set up a server (Ubuntu linux) that has been happily running backuppc for several years handling backups for all the machines in the office with grace AND style. We love it. However, in the last few months some of the users have noticed that when backuppc is running a backup (incremental or full - does not seem to matter which) it can have a serious impact to the performance of their local machine. Stuff comes to a crawl and they are nearly unable to work because simple things like switching from one application to another starts to take several seconds, etc. The machine behaves like it is hammering swap space and thrashing for memory. At least one user reports this goes on for several hours (and I confirmed that his latest incremental took 119 minutes to complete). All the machines affected in this way are wired to the gigabit network (not wireless), and I'm using rsync for the transfer method. The users with the complaints are all using OS X on late model high-end MacBook Pro laptops. Is there anything I can to to have the backups run in a more transparent manner? We are not all that concerned with speed of backup process - we're all here all day anyway, so as long as everyone gets a backup at least once a day we're happy. Are you using any 'scan on access' type of virus protection? That would be odd for a Mac, but I think there are such things. Nope, no realtime scanning stuff at all. Do any have local time machine backups that might be included? No, time machine is on external drives, specifically excluded from backups. Or directories with very large numbers of files? This I can check on. What is considered very large numbers of files? More than 1024? More than 102400? I think the rsync at each end will keep a copy of the whole directory tree in memory while both ends walk and compare contents. Normally this would be very fast on incrementals where it doesn't do more than the directory check for files that match but the list might be big enough to swap to disk. Hmm.. Does it produce a copy of the whole directory tree for each backup location? If so, would it be beneficial to split up the backups such that instead of telling it to backup /Users/myusername/ I explicitly list each of the directories in my home: /Users/myusername/Documents/ /Users/myusername/Library ...etc? Thanks! -- Kimball -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
On Jan 31, 2012, at 5:47 PM, Steve wrote:
On Tue, Jan 31, 2012 at 5:25 PM, Kimball Larsen quang...@gmail.com wrote:
Is there anything I can to to have the backups run in a more transparent
manner? We are not all that concerned with speed of backup process - we're
all here all day anyway, so as long as everyone gets a backup at least once
a day we're happy.
check the nice level during the backup (on the client) and see where
rsync is running. If it's the same as the other user processes, maybe
change the $Conf{RsyncClientCmd} to include a nice level? I know that
is suggested here:
http://backuppc.sourceforge.net/faq/ssh.html
Ah - this looks to be exactly what I'm looking for. I figured I could probably
nice the rsync command, but didn't know exactly where to find the correct
syntax.
I'll give this a whirl.
Thanks!
-- Kimball
--
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
-Original Message-
From: Steve [mailto:lepe...@gmail.com]
Sent: den 1 februari 2012 01:48
To: General list for user discussion, questions and support
Subject: Re: [BackupPC-users] Backuppc is Hammering My Clients
check the nice level during the backup (on the client) and see where
rsync is running. If it's the same as the other user processes, maybe
change the $Conf{RsyncClientCmd} to include a nice level? I know that
is suggested here:
http://backuppc.sourceforge.net/faq/ssh.html
Good call, thanks. I'll try that on a machine over here. Backups take
forever on this ancient computer...
--
/Sorin
smime.p7s
Description: S/MIME cryptographic signature
--
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
On 2012-02-01 08:32, Kimball Larsen wrote:
On Jan 31, 2012, at 5:47 PM, Steve wrote:
check the nice level during the backup (on the client) and see where
rsync is running. If it's the same as the other user processes, maybe
change the $Conf{RsyncClientCmd} to include a nice level? I know that
is suggested here:
http://backuppc.sourceforge.net/faq/ssh.html
Ah - this looks to be exactly what I'm looking for. I figured I could
probably nice the rsync command, but didn't know exactly where to find the
correct syntax.
Damn, others beat me to it. For Mac OS clients, I use:
$Conf{RsyncClientCmd} = '$sshPath -q -x -l root $host /usr/bin/nice -n 19
$rsyncPath $argList+';
Nice helps CPU priority, but it does nothing for disk IO. So for Linux
clients, I use:
$Conf{RsyncClientCmd} = '$sshPath -q -x -l root $host /usr/bin/nice -n 19
/usr/bin/ionice -c 3 $rsyncPath $argList+';
Regards,
Tyler
--
I am patient with stupidity but not with those who are proud of it.
-- Edith Sitwell
--
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
On Tue, 2012-01-31 at 19:47 -0500, Steve wrote:
On Tue, Jan 31, 2012 at 5:25 PM, Kimball Larsen quang...@gmail.com wrote:
Is there anything I can to to have the backups run in a more transparent
manner? We are not all that concerned with speed of backup process -
we're all here all day anyway, so as long as everyone gets a backup at
least once a day we're happy.
check the nice level during the backup (on the client) and see where
rsync is running. If it's the same as the other user processes, maybe
change the $Conf{RsyncClientCmd} to include a nice level? I know that
is suggested here:
http://backuppc.sourceforge.net/faq/ssh.html
If you add the flags -c arcfour to the ssh command line that will use
the arcfour cypher which is less less secure but much cpu intensive, I
use it on CPU bound clients such as my iPhone.
--
Tim Fletcher t...@night-shade.org.uk
--
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
On Wed, Feb 1, 2012 at 2:30 AM, Kimball Larsen quang...@gmail.com wrote: Do any have local time machine backups that might be included? No, time machine is on external drives, specifically excluded from backups. It might be worth checking that the excludes work and the links that make it show on the desktop aren't being followed. Or directories with very large numbers of files? This I can check on. What is considered very large numbers of files? More than 1024? More than 102400? It would be relative to the amount of RAM available - probably millions. I think the rsync at each end will keep a copy of the whole directory tree in memory while both ends walk and compare contents. Normally this would be very fast on incrementals where it doesn't do more than the directory check for files that match but the list might be big enough to swap to disk. Hmm.. Does it produce a copy of the whole directory tree for each backup location? If so, would it be beneficial to split up the backups such that instead of telling it to backup /Users/myusername/ I explicitly list each of the directories in my home: /Users/myusername/Documents/ /Users/myusername/Library ...etc? Each 'share' is processed in a separate run. It will only help if the number of files really is a problem. I always add --one-file-system to the rsync args to avoid accidentally walking into nfs mounts or external drives or DVDs that might be mounted. If you do, be sure to add shares for everything you need. -- Les Mikesell lesmikes...@gmail.com -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
Les Mikesell wrote at about 07:40:20 -0600 on Wednesday, February 1, 2012: On Wed, Feb 1, 2012 at 2:30 AM, Kimball Larsen quang...@gmail.com wrote: Do any have local time machine backups that might be included? No, time machine is on external drives, specifically excluded from backups. It might be worth checking that the excludes work and the links that make it show on the desktop aren't being followed. Or directories with very large numbers of files? This I can check on. What is considered very large numbers of files? More than 1024? More than 102400? It would be relative to the amount of RAM available - probably millions. I have no trouble backing up half a million files on a system with just 512MB. Surely, if you have high-powered relatively new PCs you will have many 4+ Gigabytes so it is unlikely that RAM swapping will be the problem. Plus any 3.0 version of rsync uses RAM quite efficiently. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
On Tue, Jan 31, 2012 at 5:25 PM, Kimball Larsen quang...@gmail.com wrote: We are a small office (6 employees) with a mixture of windows and mac machines sitting on desks. I have set up a server (Ubuntu linux) that has been happily running backuppc for several years handling backups for all the machines in the office with grace AND style. We love it. However, in the last few months some of the users have noticed that when backuppc is running a backup (incremental or full - does not seem to matter which) it can have a serious impact to the performance of their local machine. Stuff comes to a crawl and they are nearly unable to work because simple things like switching from one application to another starts to take several seconds, etc. The machine behaves like it is hammering swap space and thrashing for memory. At least one user reports this goes on for several hours (and I confirmed that his latest incremental took 119 minutes to complete). All the machines affected in this way are wired to the gigabit network (not wireless), and I'm using rsync for the transfer method. The users with the complaints are all using OS X on late model high-end MacBook Pro laptops. Is there anything I can to to have the backups run in a more transparent manner? We are not all that concerned with speed of backup process - we're all here all day anyway, so as long as everyone gets a backup at least once a day we're happy. Are you using any 'scan on access' type of virus protection? That would be odd for a Mac, but I think there are such things. Do any have local time machine backups that might be included? Or directories with very large numbers of files? I think the rsync at each end will keep a copy of the whole directory tree in memory while both ends walk and compare contents. Normally this would be very fast on incrementals where it doesn't do more than the directory check for files that match but the list might be big enough to swap to disk. -- Les Mikesell lesmikes...@gmail.com -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
On Tue, Jan 31, 2012 at 5:25 PM, Kimball Larsen quang...@gmail.com wrote:
Is there anything I can to to have the backups run in a more transparent
manner? We are not all that concerned with speed of backup process - we're
all here all day anyway, so as long as everyone gets a backup at least once
a day we're happy.
check the nice level during the backup (on the client) and see where
rsync is running. If it's the same as the other user processes, maybe
change the $Conf{RsyncClientCmd} to include a nice level? I know that
is suggested here:
http://backuppc.sourceforge.net/faq/ssh.html
A.
--
The universe is probably littered with the one-planet graves of
cultures which made the sensible economic decision that there's no
good reason to go into space--each discovered, studied, and remembered
by the ones who made the irrational decision. - Randall Munroe
--
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backuppc is Hammering My Clients
Two things :osX systemusers started to complain. Logs check of the backup server. Do parts of server fail? Disks perhaps? Why not skip the dailyy routine of one a week randomly during? Prolongs life of server (parts) steve has you looking up the faq has not he? Try adding bitsamplerate (oops! Other programme!) but seriously do a wear and tear check of the server. Logs should indicate something around the time slow became an issue -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
