Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-09 Thread Norbert Schulze
Hello Ryan, 1) Raid5X is going to kill your performance. As mentioned in a previous message, convert to RAID10 if you can afford the space decrease. The write speed of RAID 50 is higher than RAID 5 and i think this is not the problem on my system. The backuppc server input - reading from

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Norbert Schulze
What filesystem are you using? The standard filesystem of ubuntu 9.10 = EXT4 Regards Norbert -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively,

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Norbert Schulze
Using PAE, you can have 3.5 G of usable ram on a system. HOWEVER, each r...@server:~# uname -a Linux server 2.6.31-20-generic-pae #58-Ubuntu SMP Fri Mar 12 06:25:51 UTC 2010 i686 GNU/Linux Currently still running admin4/8 01:00 BackupPC_nightly -m 0 255 11896 top - 09:47:18 up 19 days,

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Tino Schwarze
On Wed, Apr 07, 2010 at 05:02:33PM -0400, Josh Malone wrote: OS is Ubuntu 9.04 32Bit IMHO it is better to migrate to a 64Bit-System!? I don't see an urgent reason to migrate to 64 bit... I would have installed this machine 64 bit at the beginning, just because it's a 64 bit machine.

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Tino Schwarze
On Wed, Apr 07, 2010 at 01:03:18PM +0200, Norbert Schulze wrote: Or just post the output of vmstat 10 10 r...@server:~# vmstat 10 10 procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si sobibo in cs us sy id wa

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Norbert Schulze
Hello Tino, Your I/O system seems saturated (70-80% of time is spent waiting for I/O). Try running only one BackupPC_nightly in parallel. Yesterday i set $Conf{MaxBackupPCNightlyJobs} = 2; to $Conf{MaxBackupPCNightlyJobs} = 1; Currently still running :-/ admin4/8 01:00 BackupPC_nightly

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Gerald Brandt
Hi, - Norbert Schulze n...@nurfuerspam.de wrote: Hello Tino, Your I/O system seems saturated (70-80% of time is spent waiting for I/O). Try running only one BackupPC_nightly in parallel. Yesterday i set $Conf{MaxBackupPCNightlyJobs} = 2; to $Conf{MaxBackupPCNightlyJobs} = 1;

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Norbert Schulze
Hello Gerald, Is this running in a VM? I often see high CPU wait times when doing disk I/O in a VM. No, it is a normal system with a hardware raid-controller. Regards Norbert -- Download Intel#174; Parallel Studio

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Daniel Berteaud
Le jeudi 08 avril 2010 à 15:36 +0200, Norbert Schulze a écrit : Hello Gerald, Is this running in a VM? I often see high CPU wait times when doing disk I/O in a VM. No, it is a normal system with a hardware raid-controller. Do you have a BBU on the raid controler ? Hardware RAID

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Les Mikesell
On 4/8/2010 8:36 AM, Norbert Schulze wrote: Hello Gerald, Is this running in a VM? I often see high CPU wait times when doing disk I/O in a VM. No, it is a normal system with a hardware raid-controller. Raid5? That has horrible write performance, especially with small writes like the

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Norbert Schulze
Hello Daniel, Do you have a BBU on the raid controler ? ADAPTEC 3805 RAID SAS/SATA 8−Kanal Hardware RAID without BBU (and of course, without write-back cache enabled) is usually *very* slow, much slower than simple disk system, or software RAID. When you look at my other postings, the

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Norbert Schulze
Raid5? That has horrible write performance, It is a RAID50 IMHO: BackupPC_nightly mainly reads from disk!? If you aren't pressed for disk space you might improve things by increasing $Conf{BackupPCNightlyPeriod} so it only traverses part I change this now to 4 Regards Norbert

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Les Mikesell
On 4/8/2010 9:48 AM, Norbert Schulze wrote: Raid5? That has horrible write performance, It is a RAID50 IMHO: BackupPC_nightly mainly reads from disk!? It traverses the pool directory, stat()'ing each inode (which isn't like to be near the directory) looking for entries that only have one

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-08 Thread Ryan Manikowski
On 4/8/2010 12:23 PM, Les Mikesell wrote: On 4/8/2010 9:48 AM, Norbert Schulze wrote: Raid5? That has horrible write performance, It is a RAID50 IMHO: BackupPC_nightly mainly reads from disk!? It traverses the pool directory, stat()'ing each inode (which isn't like to be

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-07 Thread Tino Schwarze
Hi Norbert, On Wed, Apr 07, 2010 at 11:34:57AM +0200, Norbert Schulze wrote: the BackupPC_nightly takes too much time. Is this too much data for this server? Intel(R) Core(TM)2 Quad CPUQ9650 @ 3.00GHz Memory: 8GB General Server Information The servers PID is 15477, on host

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-07 Thread Norbert Schulze
take a look at what vmstat 10 r...@server:/var/www# vmstat procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si sobibo in cs us sy id wa 1 4 14764 6235884 688976 13520400 2 104 2 1 78 18

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-07 Thread Tino Schwarze
On Wed, Apr 07, 2010 at 12:11:31PM +0200, Norbert Schulze wrote: take a look at what vmstat 10 r...@server:/var/www# vmstat procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si sobibo in cs us sy id wa 1 4

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-07 Thread Norbert Schulze
I suppose, this is the first line of vmstat output? Sorry :-) Or just post the output of vmstat 10 10 r...@server:~# vmstat 10 10 procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si sobibo in cs us sy id wa 0 5

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-07 Thread David Williams
: Re: [BackupPC-users] BackupPC_nightly takes too much time On Wed, 07 Apr 2010 11:34:57 +0200, Norbert Schulze n...@nurfuerspam.de wrote: Hello, the BackupPC_nightly takes too much time. Is this too much data for this server? Intel(R) Core(TM)2 Quad CPUQ9650 @ 3.00GHz Memory

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-07 Thread Richard Shaw
On Wed, Apr 7, 2010 at 5:29 AM, Tino Schwarze backuppc.li...@tisc.de wrote: On Wed, Apr 07, 2010 at 12:11:31PM +0200, Norbert Schulze wrote: [SNIP] OS is Ubuntu 9.04 32Bit IMHO it is better to migrate to a 64Bit-System!? I don't see an urgent reason to migrate to 64 bit... I would have

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-07 Thread Josh Malone
On Wed, 7 Apr 2010 16:47:55 -0400, David Williams dwilli...@dtw-consulting.com wrote: Josh, Interesting that. Is there an easy way to convert from ext3 to ext4? Or do you need to reformat? Also, did you change all your hard drives to ext4 or just the drive backuppc backs up to? It can be

Re: [BackupPC-users] BackupPC_nightly takes too much time

2010-04-07 Thread Josh Malone
On Wed, 7 Apr 2010 15:56:19 -0500, Richard Shaw hobbes1...@gmail.com wrote: On Wed, Apr 7, 2010 at 5:29 AM, Tino Schwarze backuppc.li...@tisc.de wrote: On Wed, Apr 07, 2010 at 12:11:31PM +0200, Norbert Schulze wrote: [SNIP] OS is Ubuntu 9.04 32Bit IMHO it is better to migrate to a