Tim,
First off, thank you very much for your insightful and helpful approach,
despite the limited information I provided. And thank you as well for being the
“unpaid” mechanic (good analogy). I will try to present a detail account.
Please read my comments to your questions below:
Environment:
· Server (using gigabit wired Ethernet on a managed switch)
o running under VMWare ESX5.5
o VM allocated w/ 4gb ram, all procs/cores, and max cpu/mem bursts
o intel i5-3330 3GHz
o 24gb ram
o Backup location 5 disk hardware raid 5 under Centos 6.5
§ Backup avail space 6 TB
· Windows Workstation Client (using gigabit wired Ethernet on a managed
switch)
o Intel i7-4770K cpu 3.5GHz
o 32 gb ram
o 2 internal logical drives
§ C: HW raid 0 comprising of TWO 120GB SSD drives (238gb capacity)
· 89.5GB used (162,337 files)
§ D: HW raid 5 consisting of 4 x 2TB (5589gb capacity)
· 1.5TB used (213,594 files)
I have not completed a full backup of this workstation yet. I have done many
incremental removing exclusions in an attempt to incrementally backup the
entire computer (drive c &d) in steps.
I will disable compression and re-attempt, although I will not be able to
report the information for xx days until its complete.
> Now, these are established backups. I've found that initial backups can be
> 2-3 times as long. And at 3TB, you're nearly triple of the bigger one. So,
> that could mean that the initial backup can take 6000
> minutes or more, or four days. And personally, I've found that compression
> can double or triple the time again. Worst-case of all of that could be two
> weeks! So it is possible that you're seeing correct
> operation for that first backup, and that future backups should be much
> better.
>
So I guess compression is out. I will disable. Thank you for the info.
> Also, those are fulls. Incrementals take 4 hours and 1 hour, respectively.
>
> Is rsync “really†that slow?
>
> No: rsync is only going to be marginally slower than a non-rsync copy, even
> on the first time, assuming you're not limited by something else (CPU or RAM)
> that would not be a limit for a normal copy.
>
> That could be related to the number of files: that's an area where rsync can
> get tripped up. As you can see, I've got >1 million files, so the definition
> of "too many" is pretty big. But if you had, say, 10M files,
> maybe that's an issue to consider.
I don’t have anywhere near a million files, so I guess my workstation should be
quicker. I suspect vmware is causing *some* penalty, but it should not be that
great. Regardless, it’s what I have so that’s got to work fine.
> Yes. First, you've given us *NOTHING* to go on other than "it's slow". It's
> like telling a mechanic "my car doesn't run right." Of course, you're
> probably expecting to pay the mechanic, so he's incented to ask
> lots of questions to figure things out. I think it's pretty telling that I
> can't see a single reply to your request. While your request includes a lot
> of words, there was almost *NOTHING* of substance in your
> request except a *VERY* brief description of your hardware. We have almost
> no description of what your backups look like (size and number of files, for
> example), what your backup jobs look like (compression
> being a very big one), or what your backup server is doing (CPU, memory or
> disk usage). And we're not getting paid, so we're not really incented to
> ask you a lot of questions. But I'll give you a few:
>
Yes, I’m sorry. Your points are well made. I’m trying to correct my mistake in
this email.
> First, is your system limited in some way? Are you hitting 100% CPU, or
> swapping (not enough RAM), or are your disk overloaded, or something else?
> Learn to use top and vmstat (or dstat).
CPU normally sits around 50%-70% depending if a 2nd job kicks in. I’ve used
“iostat -k 5” and %iowait from 0% to as much as 40% but usualy hovers around
10%-20%. I was unaware of dstat, but I’ll start running it when I perform the
full backup
While doing some googling, I read somewhere about adjusting the TCP buffers for
rsync. I’ve applied the following line in my rsync.conf file on the windows
machine:
socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=65536
Have you experimented with this or something similar in order to improve
performance? I don’t have any hard numbers with/without (yet), but thought I
would mention this.
> Is your infrastructure fundamentally limited in some way? Have you tried
> doing a straight-up copy from a target to your backup system to make sure
> that the underlying infrastructure is capable of delivering
> what you expect it to? If you can only get 1-2MB/s copies using SMB, tar,
> NFS, FTP, etc. , then that's all you'll get with BackupPC, too. But if you
> can get 70MB/s copies between the same two systems some
> other way, then we can expect better of BackupPC. (But all that does is
> re-ask the question of what is limiting you.)
No, my baseline was based on my prior experience with bacula, and a trial of
Retrospect (which I quickly discounted). Both software I was able to fully back
up this workstation within 4-8 hrs—not days. This is what started to get me
worried.
> From my e-mail, you know what is possible and reasonable to get. If you're
> far away from those results, then you need to figure out what is different
> about your system and causing the slowdown.
> The second thing to try is to simplify things. For me, the first thing I do
> is disable compression. In today's multi-core universe, compression is
> rapidly becoming a bottleneck again. The compression algorithms
> in common use today do *not* use multiple cores. On a system with more than
> a couple of disks I can easily max out one core with compression.
Yes, I learned this while experimenting between gzip and pigz. For now, I’ll
disable compression.
> Another option would be to expand your testing from a very small section to
> something larger: say, 100GB. That is big enough to be somewhat
> representative of the whole, but should be able to complete
> quickly enough even with compression, encryption, etc. to get some baseline
> numbers to work with, including both the *first* and *additional* full
> backups. That way, you might find that the initial backup will
> take a week, but each additional backup after that will only take 12 hours
> and you're OK with that. Or, you might find that things are still broken,
> but now it won't cost you a week of your life every time you
> want to test.
GREAT idea. Before I embark on trying to do a full backup, I’ll isolate ONE
folder with some large files (around 100gb) and focus on doing some tests on
that with/without compression. I will report my findings.
> If it helps, the server is beefy, quad-core i5 proc w/ 4gb ram and 6TB raid5.
Tim, again, thank you for taking the time and interest to both help and point
out the vagueness of the email. I will get back with you.
Thanks,
Marco
------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
