Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Frédéric Massot
Le 17/09/2012 17:01, Tyler J. Wagner a écrit : On 2012-09-17 14:18, Frédéric Massot wrote: Is the relatime option is acceptable to replace atime? Unless you are using mutt on the BackupPC server, use noatime. There is no longer any common use for file access time. Hi, From what I've read,

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
Les Mikesell lesmikes...@gmail.com wrote on 09/17/2012 01:34:33 PM: On Mon, Sep 17, 2012 at 11:05 AM, Timothy J Massey tmas...@obscorp.com wrote: I'm writing a longer reply, but here's a quick in-thread reply: I know exactly what you mean by waiting until after the first full. Often

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012 01:30:36 PM: AFAIK this is not correct. If checksum caching is enabled, backuppc will check the cached checksums against the actual file contents based on the setting of the: $Conf{RsyncCsumCacheVerifyProb} = 0.10; Yeah, I

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Mark Coetser
On 17/09/2012 17:16, Mark Coetser wrote: On 17/09/2012 17:01, Les Mikesell wrote: On Mon, Sep 17, 2012 at 7:59 AM, Mark Coetserm...@tux-edo.co.za wrote: Surely disk io would affect normal rsync as well? Normal rsync and even nfs get normal transfer speeds its only rsync within backuppc that

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012 02:05:28 PM: On Mon, Sep 17, 2012 at 12:54:35PM -0400, Timothy J Massey wrote: No matter the size of the system, I seem to top out at about 50GB/hour for full backups. Here is a perfectly typical example: Full Backup:

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Tyler J. Wagner
On 2012-09-18 15:09, Frédéric Massot wrote: From what I've read, the new kernel mount the ext4 filesystem with acl and relatime options by default. So for backuppc, add options noacl, norelatime and noatime in fstab. I do not know if noatime implies norelatime? There is no norelatime as

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
Tim Fletcher t...@night-shade.org.uk wrote on 09/17/2012 08:50:39 AM: You are being hit by disk io speeds, check you dont have atime turned on on the fs. I agree that noatime is a net win for *very* little pain. I found a system where I had not mounted the datastore noatime and swiched it.

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012 02:33:34 PM: I have another system that is lower power: 2632652.8 MB at 662.5 minutes or 66MB/s That's 2.6TB in 11 hours. That is perfectly acceptable for me. And *way* better than I'm getting. the BackupPC system is a 4

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
Les Mikesell lesmikes...@gmail.com wrote on 09/17/2012 02:44:20 PM: On Mon, Sep 17, 2012 at 11:54 AM, Timothy J Massey tmas...@obscorp.com wrote: However, I have recently inherited a server that is 3TB big, and 97% full, too! Backups of that system take 3.5 *days* to complete. I

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Frédéric Massot
Le 18/09/2012 16:58, Tyler J. Wagner a écrit : On 2012-09-18 15:09, Frédéric Massot wrote: From what I've read, the new kernel mount the ext4 filesystem with acl and relatime options by default. So for backuppc, add options noacl, norelatime and noatime in fstab. I do not know if noatime

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
Mark Coetser m...@tux-edo.co.za wrote on 09/18/2012 10:21:42 AM: I am busy running a full clean rsync to time exactly how long it will take and will post results compared to a clean full backup with backuppc, I can tell you that the network interface on the backup server is currently

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Les Mikesell
On Tue, Sep 18, 2012 at 9:53 AM, Timothy J Massey tmas...@obscorp.com wrote: All of that was clearly outlined at the top of the e-mail: 4 x 2TB Seagate SATA drives in RAID-5 (using md, which I''m not sure I stated originally). Raid5 has its own reasons for bad write performance. You

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Les Mikesell
On Tue, Sep 18, 2012 at 10:01 AM, Timothy J Massey tmas...@obscorp.com wrote: I still have this on the (very) back burner. It's a Windows system, so I'm looking at SMB, and I *hate* backups over SMB. They are nothing but a problem. I'd expect cygwin tar over ssh to work on windows, but

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Les Mikesell
On Tue, Sep 18, 2012 at 10:24 AM, Timothy J Massey tmas...@obscorp.com wrote: Fortunately, BackupPC is a backup of the backup right now, and is not expected to be used for real. Yet. That's why I can take the time and try to actually solve the problem, rather than apply band-aids. But that

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Les Mikesell
On Tue, Sep 18, 2012 at 9:21 AM, Mark Coetser m...@tux-edo.co.za wrote: I am busy running a full clean rsync to time exactly how long it will take and will post results compared to a clean full backup with backuppc, I can tell you that the network interface on the backup server is currently

[BackupPC-users] BackupPC is awesome

2012-09-18 Thread Adam Goryachev
Hi all, Just wanted to point out how great backuppc is. I have one backuppc system configured to backup daily, and never expire a backup. # hosts 13 # Full backups 1243 # Incremental backups 7417 Oldest backup 2223.7 days old (backup #0, still not retired) * Uncompressed pool: o Pool

Re: [BackupPC-users] BackupPC is awesome

2012-09-18 Thread Timothy J Massey
Adam Goryachev mailingli...@websitemanagers.com.au wrote on 09/18/2012 12:58:40 PM: I just wanted to put out there, that backuppc can work really well at backing up, and archiving data. In case there was doubt, I want to echo this: I have 6-8 BackupPC servers in production in various

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
Timothy J Massey tmas...@obscorp.com wrote on 09/18/2012 11:07:18 AM: John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012 02:33:34 PM: I have another system that is lower power: 2632652.8 MB at 662.5 minutes or 66MB/s That's 2.6TB in 11 hours. That is perfectly

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
Les Mikesell lesmikes...@gmail.com wrote on 09/18/2012 12:42:26 PM: On Tue, Sep 18, 2012 at 10:24 AM, Timothy J Massey tmas...@obscorp.com wrote: Fortunately, BackupPC is a backup of the backup right now, and is not expected to be used for real. Yet. That's why I can take the time

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Les Mikesell
On Tue, Sep 18, 2012 at 1:14 PM, Timothy J Massey tmas...@obscorp.com wrote: I have just performed some full backups on a host after disabling compression. Results: Not-first backup with compression: 139.9 minutes for 7.6 MB First backup without compression: 76.7 minutes for 70181.5

[BackupPC-users] Backup aborted (Tar exited with error 512 () status)

2012-09-18 Thread Dave Williams
All (trying again as no response to date), Started to get this error during my backup. I have looked through the log files and I can't find anything that points to what is causing this error. Only thing that I assume is that it's happening in /photos/Laptop1, but not really sure. How can I

Re: [BackupPC-users] Backup aborted (Tar exited with error 512 () status)

2012-09-18 Thread Carl Wilhelm Soderstrom
On 09/18 02:55 , Dave Williams wrote: Ran the following command and that worked just fine: tar -cvf - /photos/Laptop1 |cat /dev/null 2012-09-13 14:49:34 incr backup started back to 2012-08-11 22:55:43 (backup #614) for directory /photos/Laptop1 2012-09-13 15:50:26 Got fatal error during

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Timothy J Massey
Les Mikesell lesmikes...@gmail.com wrote on 09/18/2012 03:34:56 PM: On Tue, Sep 18, 2012 at 1:18 PM, Timothy J Massey tmas...@obscorp.com wrote: That is a good point, but if I ever have to do a full 3TB restore from BackupPC, the 12 hours a (properly performing) BackupPC will take is not

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread Les Mikesell
On Tue, Sep 18, 2012 at 3:50 PM, Timothy J Massey tmas...@obscorp.com wrote: Will do. You were the one that turned me on to Clonezilla, and I'm always open to new tools... :) ReaR is not exactly the most Google-friendly search term (on so many levels...), so for others (and to confirm):

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-18 Thread backuppc
Timothy J Massey wrote at about 12:54:35 -0400 on Monday, September 17, 2012: I have several very similar configurations. Here's an example: Atom D510 (1.66GHz x 2 Cores) 4GB RAM CentOS 6 64-bit 4 x 2TB Seagate SATA drives in RAID-6 configuration I get almost 200 MB/s