On 19/09/2012 11:59, Mark Coetser wrote:
On 19/09/2012 11:21, Mark Coetser wrote:
sent 150123974 bytes received 1745633674066 bytes 13566176.70 bytes/sec
total size is 2663529076313 speedup is 1.53
real2144m46.748s
user144m11.809s
sys 668m4.861s
Looks like the above was
Les Mikesell lesmikes...@gmail.com wrote on 09/18/2012 07:04:21 PM:
The guest servers are not hurting for resources. They are not part of
the
problem. The problem seems to be contained completely inside of the
BackupPC server.
If you aren't seeing big speed differences among clients
On Fri, Sep 21, 2012 at 11:46 AM, Timothy J Massey tmas...@obscorp.com wrote:
I would not expect the network speed changing from 100Mb to 1Gb to make a
difference: this server is literally unchanged from one run to the next.
BackupPC literally shows zero files changed! So the network
On Fri, Sep 21, 2012 at 12:11:20PM -0500, Les Mikesell wrote:
On Fri, Sep 21, 2012 at 11:46 AM, Timothy J Massey tmas...@obscorp.com
wrote:
I would not expect the network speed changing from 100Mb to 1Gb to make a
difference: this server is literally unchanged from one run to the next.
On Fri, Sep 21, 2012 at 1:02 PM, John Rouillard
rouilj-backu...@renesys.com wrote:
However, I just run into a situation in an rsync restore where both
ends were sitting in a select() apparently waiting for each other for
so long I gave up and used a tar download. Maybe there is a bug
On 18/09/2012 17:34, Timothy J Massey wrote:
Mark Coetser m...@tux-edo.co.za wrote on 09/18/2012 10:21:42 AM:
I am busy running a full clean rsync to time exactly how long it will
take and will post results compared to a clean full backup with
backuppc, I can tell you that the network
On 19/09/2012 11:21, Mark Coetser wrote:
On 18/09/2012 17:34, Timothy J Massey wrote:
Mark Coetserm...@tux-edo.co.za wrote on 09/18/2012 10:21:42 AM:
I am busy running a full clean rsync to time exactly how long it will
take and will post results compared to a clean full backup with
On Tue, Sep 18, 2012 at 11:07:18AM -0400, Timothy J Massey wrote:
John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012 02:33:34
PM:
I have another system that is lower power:
2632652.8 MB at 662.5 minutes or 66MB/s
That's 2.6TB in 11 hours. That is perfectly acceptable
On Tue, Sep 18, 2012 at 10:53:52AM -0400, Timothy J Massey wrote:
John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012 02:05:28
PM:
On Mon, Sep 17, 2012 at 12:54:35PM -0400, Timothy J Massey wrote:
My backup drive is a 1U linux box exporting its disks as an isci
system running
On Wed, Sep 19, 2012 at 10:45 AM, Timothy J Massey tmas...@obscorp.com wrote:
Can you repeat those the 3rd time with checksum-caching enabled so we
have a better idea of how much time that saves? (If you didn't have
it on already, you'd need to repeat again to store them first)
The host
backu...@kosowsky.org wrote on 09/18/2012 09:51:11 PM:
Timothy J Massey wrote at about 12:54:35 -0400 on Monday, September 17,
2012:
I have several very similar configurations. Here's an example:
Atom D510 (1.66GHz x 2 Cores)
4GB RAM
CentOS 6 64-bit
4 x 2TB Seagate SATA
On Mon, Sep 17, 2012 at 12:22:23PM -0300, Rodrigo Severo wrote:
On Mon, Sep 17, 2012 at 12:16 PM, Mark Coetser m...@tux-edo.co.za wrote:
Its the first full run but its taking forever to complete, it was
running for nearly 3 days!
I'm seeing similar issues here.
Is there any troubleshooting
Le 17/09/2012 17:01, Tyler J. Wagner a écrit :
On 2012-09-17 14:18, Frédéric Massot wrote:
Is the relatime option is acceptable to replace atime?
Unless you are using mutt on the BackupPC server, use noatime. There is no
longer any common use for file access time.
Hi,
From what I've read,
Les Mikesell lesmikes...@gmail.com wrote on 09/17/2012 01:34:33 PM:
On Mon, Sep 17, 2012 at 11:05 AM, Timothy J Massey
tmas...@obscorp.com wrote:
I'm writing a longer reply, but here's a quick in-thread reply:
I know exactly what you mean by waiting until after the first full.
Often
John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012 01:30:36
PM:
AFAIK this is not correct. If checksum caching is enabled, backuppc
will check the cached checksums against the actual file contents based
on the setting of the:
$Conf{RsyncCsumCacheVerifyProb} = 0.10;
Yeah, I
On 17/09/2012 17:16, Mark Coetser wrote:
On 17/09/2012 17:01, Les Mikesell wrote:
On Mon, Sep 17, 2012 at 7:59 AM, Mark Coetserm...@tux-edo.co.za wrote:
Surely disk io would affect normal rsync as well? Normal rsync and even
nfs get normal transfer speeds its only rsync within backuppc that
John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012 02:05:28
PM:
On Mon, Sep 17, 2012 at 12:54:35PM -0400, Timothy J Massey wrote:
No matter the size of the system, I seem to top out at about 50GB/hour
for
full backups. Here is a perfectly typical example:
Full Backup:
On 2012-09-18 15:09, Frédéric Massot wrote:
From what I've read, the new kernel mount the ext4 filesystem with acl
and relatime options by default. So for backuppc, add options noacl,
norelatime and noatime in fstab.
I do not know if noatime implies norelatime?
There is no norelatime as
Tim Fletcher t...@night-shade.org.uk wrote on 09/17/2012 08:50:39 AM:
You are being hit by disk io speeds, check you dont have atime
turned on on the fs.
I agree that noatime is a net win for *very* little pain. I found a
system where I had not mounted the datastore noatime and swiched it.
John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012 02:33:34
PM:
I have another system that is lower power:
2632652.8 MB at 662.5 minutes or 66MB/s
That's 2.6TB in 11 hours. That is perfectly acceptable for me. And *way*
better than I'm getting.
the BackupPC system is a 4
Les Mikesell lesmikes...@gmail.com wrote on 09/17/2012 02:44:20 PM:
On Mon, Sep 17, 2012 at 11:54 AM, Timothy J Massey
tmas...@obscorp.com wrote:
However, I have recently inherited a server that is 3TB big, and 97%
full, too! Backups of that system take 3.5 *days* to complete. I
Le 18/09/2012 16:58, Tyler J. Wagner a écrit :
On 2012-09-18 15:09, Frédéric Massot wrote:
From what I've read, the new kernel mount the ext4 filesystem with acl
and relatime options by default. So for backuppc, add options noacl,
norelatime and noatime in fstab.
I do not know if noatime
Mark Coetser m...@tux-edo.co.za wrote on 09/18/2012 10:21:42 AM:
I am busy running a full clean rsync to time exactly how long it will
take and will post results compared to a clean full backup with
backuppc, I can tell you that the network interface on the backup server
is currently
On Tue, Sep 18, 2012 at 9:53 AM, Timothy J Massey tmas...@obscorp.com wrote:
All of that was clearly outlined at the top of the e-mail: 4 x 2TB
Seagate SATA drives in RAID-5 (using md, which I''m not sure I stated
originally).
Raid5 has its own reasons for bad write performance. You
On Tue, Sep 18, 2012 at 10:01 AM, Timothy J Massey tmas...@obscorp.com wrote:
I still have this on the (very) back burner. It's a Windows system, so
I'm looking at SMB, and I *hate* backups over SMB. They are nothing but a
problem.
I'd expect cygwin tar over ssh to work on windows, but
On Tue, Sep 18, 2012 at 10:24 AM, Timothy J Massey tmas...@obscorp.com wrote:
Fortunately, BackupPC is a backup of the backup right now, and is not
expected to be used for real. Yet. That's why I can take the time and try
to actually solve the problem, rather than apply band-aids.
But that
On Tue, Sep 18, 2012 at 9:21 AM, Mark Coetser m...@tux-edo.co.za wrote:
I am busy running a full clean rsync to time exactly how long it will
take and will post results compared to a clean full backup with
backuppc, I can tell you that the network interface on the backup server
is currently
Timothy J Massey tmas...@obscorp.com wrote on 09/18/2012 11:07:18 AM:
John Rouillard rouilj-backu...@renesys.com wrote on 09/17/2012
02:33:34 PM:
I have another system that is lower power:
2632652.8 MB at 662.5 minutes or 66MB/s
That's 2.6TB in 11 hours. That is perfectly
Les Mikesell lesmikes...@gmail.com wrote on 09/18/2012 12:42:26 PM:
On Tue, Sep 18, 2012 at 10:24 AM, Timothy J Massey
tmas...@obscorp.com wrote:
Fortunately, BackupPC is a backup of the backup right now, and is
not
expected to be used for real. Yet. That's why I can take the time
On Tue, Sep 18, 2012 at 1:14 PM, Timothy J Massey tmas...@obscorp.com wrote:
I have just performed some full backups on a host after disabling
compression. Results:
Not-first backup with compression: 139.9 minutes for 7.6 MB
First backup without compression: 76.7 minutes for 70181.5
Les Mikesell lesmikes...@gmail.com wrote on 09/18/2012 03:34:56 PM:
On Tue, Sep 18, 2012 at 1:18 PM, Timothy J Massey tmas...@obscorp.com
wrote:
That is a good point, but if I ever have to do a full 3TB restore from
BackupPC, the 12 hours a (properly performing) BackupPC will take is
not
On Tue, Sep 18, 2012 at 3:50 PM, Timothy J Massey tmas...@obscorp.com wrote:
Will do. You were the one that turned me on to Clonezilla, and I'm always
open to new tools... :)
ReaR is not exactly the most Google-friendly search term (on so many
levels...), so for others (and to confirm):
Timothy J Massey wrote at about 12:54:35 -0400 on Monday, September 17, 2012:
I have several very similar configurations. Here's an example:
Atom D510 (1.66GHz x 2 Cores)
4GB RAM
CentOS 6 64-bit
4 x 2TB Seagate SATA drives in RAID-6 configuration
I get almost 200 MB/s
Hi
backuppc 3.1.0-9.1
rsync 3.0.7-2
OK I have a fairly decent spec backup server with 2 gigabit e1000 nics
bonned together and running in bond mode 0 all working 100%. If I run
plain rsync between the backup server
You are being hit by disk io speeds, check you dont have atime turned on on the
fs. Also it's worth considering tar instead of rsync for this sort of work
load.
--
Sent from a mobile device
Tim Fletcher
On 17 Sep 2012, at 10:08, Mark Coetser m...@tux-edo.co.za wrote:
Hi
backuppc
On 17/09/2012 14:50, Tim Fletcher wrote:
You are being hit by disk io speeds, check you dont have atime turned on on
the fs. Also it's worth considering tar instead of rsync for this sort of
work load.
--
Hi
Surely disk io would affect normal rsync as well? Normal rsync and even
nfs get
Le 17/09/2012 14:50, Tim Fletcher a écrit :
You are being hit by disk io speeds, check you dont have atime turned
on on the fs. Also it's worth considering tar instead of rsync for
this sort of work load.
Hi,
Is the relatime option is acceptable to replace atime?
Regards.
--
On 2012-09-17 14:18, Frédéric Massot wrote:
Is the relatime option is acceptable to replace atime?
Unless you are using mutt on the BackupPC server, use noatime. There is no
longer any common use for file access time.
Regards,
Tyler
--
We should forget about small efficiencies, say about 97%
On Mon, Sep 17, 2012 at 7:59 AM, Mark Coetser m...@tux-edo.co.za wrote:
Surely disk io would affect normal rsync as well? Normal rsync and even
nfs get normal transfer speeds its only rsync within backuppc that is slow.
Backuppc uses its own rsync implementation in perl on the server side
so
On 17/09/2012 17:01, Les Mikesell wrote:
On Mon, Sep 17, 2012 at 7:59 AM, Mark Coetserm...@tux-edo.co.za wrote:
Surely disk io would affect normal rsync as well? Normal rsync and even
nfs get normal transfer speeds its only rsync within backuppc that is slow.
Backuppc uses its own rsync
On Mon, Sep 17, 2012 at 10:16 AM, Mark Coetser m...@tux-edo.co.za wrote:
Its the first full run but its taking forever to complete, it was running
for nearly 3 days!
As long is it makes it through, don't make any judgements until after
the 3nd full, and be sure you have set up checksum
Les Mikesell lesmikes...@gmail.com wrote on 09/17/2012 11:51:09 AM:
On Mon, Sep 17, 2012 at 10:16 AM, Mark Coetser m...@tux-edo.co.za
wrote:
Its the first full run but its taking forever to complete, it was
running
for nearly 3 days!
As long is it makes it through, don't make any
On Mon, Sep 17, 2012 at 12:16 PM, Mark Coetser m...@tux-edo.co.za wrote:
Its the first full run but its taking forever to complete, it was
running for nearly 3 days!
I'm seeing similar issues here.
Is there any troubleshooting recommended to this kind of problem?
Rodrigo
Mark Coetser m...@tux-edo.co.za wrote on 09/17/2012 03:08:49 AM:
Hi
backuppc 3.1.0-9.1
rsync 3.0.7-2
OK I have a fairly decent spec backup server with 2 gigabit e1000 nics
bonned together and running in bond
Les Mikesell lesmikes...@gmail.com wrote on 09/17/2012 11:01:25 AM:
On Mon, Sep 17, 2012 at 7:59 AM, Mark Coetser m...@tux-edo.co.za
wrote:
Surely disk io would affect normal rsync as well? Normal rsync and
even
nfs get normal transfer speeds its only rsync within backuppc that is
slow.
Mark Coetser m...@tux-edo.co.za wrote on 09/17/2012 11:16:29 AM:
Its the first full run but its taking forever to complete, it was
running for nearly 3 days!
*IF* the backup is bandwidth-limited, the first run will take longer than
subsequent runs. How much depends on how bandwidth-limited
Rodrigo Severo rodr...@fabricadeideias.com wrote on 09/17/2012 11:22:23
AM:
On Mon, Sep 17, 2012 at 12:16 PM, Mark Coetser m...@tux-edo.co.za
wrote:
Its the first full run but its taking forever to complete, it was
running for nearly 3 days!
I'm seeing similar issues here.
Is there
On Mon, Sep 17, 2012 at 11:05 AM, Timothy J Massey tmas...@obscorp.com wrote:
I'm writing a longer reply, but here's a quick in-thread reply:
I know exactly what you mean by waiting until after the first full. Often
the second full will be faster -- but only *IF* you are bandwidth limited
On Mon, Sep 17, 2012 at 12:58:28PM -0400, Timothy J Massey wrote:
Les Mikesell lesmikes...@gmail.com wrote on 09/17/2012 11:01:25 AM:
On the 2nd it will read/uncompress
everything for block-checksum verification. If you have enabled
checksum caching, fulls after the 2nd will not have to
On Mon, Sep 17, 2012 at 12:54:35PM -0400, Timothy J Massey wrote:
No matter the size of the system, I seem to top out at about 50GB/hour for
full backups. Here is a perfectly typical example:
Full Backup: 769.3 minutes for 675677.3MB of data. That works out to be
878MB/min, or about
On Mon, Sep 17, 2012 at 06:05:28PM +, John Rouillard wrote:
On Mon, Sep 17, 2012 at 12:54:35PM -0400, Timothy J Massey wrote:
No matter the size of the system, I seem to top out at about 50GB/hour for
full backups. Here is a perfectly typical example:
Full Backup: 769.3 minutes
On Mon, Sep 17, 2012 at 11:54 AM, Timothy J Massey tmas...@obscorp.com wrote:
However, I have recently inherited a server that is 3TB big, and 97%
full, too! Backups of that system take 3.5 *days* to complete. I *can't*
live with that. I need better performance.
The only quick-fix would be
No it won't in the same way, you are basically asking rsync to walk the large
and complex file tree checking the date of every file, where as with a full
rsync all you are asking for is next file, next lie, next file
--
Sent from a mobile device
On 17 Sep 2012, at 15:59, Mark Coetser
53 matches
Mail list logo
