[Bacula-users] Windows client performance
Bacula sucks the vital essence from your computer says the slogan... and somehow it now looks true to me (though not the way it was meant to) After years of some experience with LinuxBacula combination, I finally started making some experiments with Windows client. After a few very (not so nicely) surprising experiencies in production environment, I set up a testbech with oldish 2.8 GHz (800 MHz FSB) Dual-Core P4 and 1 GB of memory, pata-interfaced 80 GB had drive, gigabit ethernet. Testbench was running a clean Windows XP sp3 with Winbacula 3.0.3a. No antivirus stuff (yet), no real applications, nothing to cause any extra mess. Fileset contains basically /WINDOWS/system32 directory. It's something I could find from every system I tried before this testbench. The very basic test result: Elapsed time: 2 mins 35 secs Priority: 10 FD Files Written: 4,648 SD Files Written: 4,648 FD Bytes Written: 664,739,011 (664.7 MB) SD Bytes Written: 665,458,318 (665.4 MB) Rate: 4288.6 KB/s Software Compression: None VSS:yes Encryption: no Accurate: no Then, after adding lowest-level (GZIP1) compression: Elapsed time: 2 mins 43 secs Priority: 10 FD Files Written: 4,648 SD Files Written: 4,648 FD Bytes Written: 308,509,378 (308.5 MB) SD Bytes Written: 309,228,685 (309.2 MB) Rate: 1892.7 KB/s Software Compression: 53.6 % VSS:yes Encryption: no Accurate: no Basically, it takes the same amount of real time to run the same job, and some more cpu load, but no peak in workstation's cpu load meter exceeded 50%. Any ideas what is the bottleneck here? I think 1GB of memory should be enough for XP alone. Some glue makes me think about the disk, but this rate is a decade below what to expect from even that old hard disk. Finally, for reference purposes I installed antivirus sw (NOD32) to the testbench, it drops only about 10% of the rate above: Elapsed time: 2 mins 51 secs Priority: 10 FD Files Written: 4,657 SD Files Written: 4,657 FD Bytes Written: 665,372,665 (665.3 MB) SD Bytes Written: 666,093,882 (666.0 MB) Rate: 3891.1 KB/s Software Compression: None VSS:yes Encryption: no Accurate: no My Bacula DIR/SD is a CentOS 5.4 box, old 32-bit piece of junk, a lower speed hw than the workstation described above. As a reference, below is a sample of backup from a linux box (64-bit CentOS), it's of the same class with rates when backing up the SD/DIR machine itself -which obviously limits the rate to 11-12MB/s level. Anyway, the samples above were clearly not limited by the Bacula server. Elapsed time: 8 mins 1 sec Priority: 10 FD Files Written: 7 SD Files Written: 7 FD Bytes Written: 5,311,971,015 (5.311 GB) SD Bytes Written: 5,311,971,759 (5.311 GB) Rate: 11043.6 KB/s Software Compression: None VSS:no Encryption: no Accurate: no So, what's that tough with my Windows clients? -- TiN -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
Timo,. Your test involved just 600 some MB of data which may not be a large or varied enough data set. There is data and then there is data. Millions of small email index files is harder on Bacula than thousands of large files. And you don't mentioned if or how you controlled for your network bandwidth - did you keep all traffic off it? I'd suggest you use a data set which is typical of your intended use and multiply it by a factor of 10 and keep bandwidth stable. Mehma === -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
On 1/25/2010 11:07 AM, Timo Neuvonen wrote: some more cpu load, but no peak in workstation's cpu load meter exceeded 50%. In Windows, 100% load means all CPU's together at max load. If you have two cores, 50% means 100% load on one core. So you're seeing the best that CPU can do (and P4's are a seriously awful design), short of a major change in the compression library to utilize multiple threads. -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
Mike Ruskai than...@earthlink.net kirjoitti viestissä news:4b5dcf2f.9050...@earthlink.net... On 1/25/2010 11:07 AM, Timo Neuvonen wrote: some more cpu load, but no peak in workstation's cpu load meter exceeded 50%. In Windows, 100% load means all CPU's together at max load. If you have two cores, 50% means 100% load on one core. Actually, there were two graphs, one per core. Neither of them peaked close to 50% at GZIP1 level. After increasing comprerssion level (up to GZIP9), it needed more cpu load, but it wasn't absolutely the limiting factor if the graphs are somehow close to truth. But the basic question still is, what limits the data rate without compression, when cpu load is very low? Using compression still drops the compressed rate by same factor the data size shrinks, while the required real time doesn't practically chage at all. Sounds very much like it was a limit of disk io rate, but it sounds unbelievable to me since figures are that low. But maybe I'll need to test disk read performance first by some other means. It will just take a couple of days until I'll have time for that project again. So you're seeing the best that CPU can do (and P4's are a seriously awful design), short of a major change in the compression library to utilize multiple threads. -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
mehma sarja mehmasa...@gmail.com kirjoitti viestissä news:ec5d34681001250830h9cfb180naea17beb1a544...@mail.gmail.com... Timo,. Your test involved just 600 some MB of data which may not be a large or varied enough data set. There is data and then there is data. Millions of small email index files is harder on Bacula than thousands of large files. And you don't mentioned if or how you controlled for your network bandwidth - did you keep all traffic off it? I'd suggest you use a data set which is typical of your intended use and multiply it by a factor of 10 and keep bandwidth stable. Network load definetely wasn't the reason here, hard to document it, but I just know it. Also, the results were very repeatable when I run the same test several times, this doesn't sound like a network load problem either. But yes, I'll need to re-run the test with another type of data. A lot of small files will cause plenty of overhead. I just didn't think it would be that much overhead... -- Timo -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
On 1/25/2010 2:46 PM, Timo Neuvonen wrote: Mike Ruskaithan...@earthlink.net kirjoitti viestissä news:4b5dcf2f.9050...@earthlink.net... On 1/25/2010 11:07 AM, Timo Neuvonen wrote: some more cpu load, but no peak in workstation's cpu load meter exceeded 50%. In Windows, 100% load means all CPU's together at max load. If you have two cores, 50% means 100% load on one core. Actually, there were two graphs, one per core. Neither of them peaked close to 50% at GZIP1 level. After increasing comprerssion level (up to GZIP9), it needed more cpu load, but it wasn't absolutely the limiting factor if the graphs are somehow close to truth. You can't go by Task Manager graphs, either. Unless you set process affinity, the thread will be scheduled on alternating cores, and the graphs won't accurately represent how loaded each core is over time. If you have a single thread running at full speed, it will very likely look like roughly 50% usage on each core over time. The calculations just don't happen with a granularity small enough to show the actual usage. If you look at the CPU usage figure on the bottom left, however, that will be reasonably accurate. If it reads 50%, then the compression work is going as fast as it can. But the basic question still is, what limits the data rate without compression, when cpu load is very low? Using compression still drops the compressed rate by same factor the data size shrinks, while the required real time doesn't practically chage at all. Sounds very much like it was a limit of disk io rate, but it sounds unbelievable to me since figures are that low. But maybe I'll need to test disk read performance first by some other means. It will just take a couple of days until I'll have time for that project again. It could be either disc I/O or network I/O that's bottlenecking. The Bacula file daemon for Windows is definitely not restricted to the rates you're seeing. -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
Elapsed time: 2 mins 35 secs Priority: 10 FD Files Written: 4,648 SD Files Written: 4,648 FD Bytes Written: 664,739,011 (664.7 MB) SD Bytes Written: 665,458,318 (665.4 MB) Rate: 4288.6 KB/s Software Compression: None VSS:yes Encryption: no Accurate: no How long does it take to copy those files to a share on another windows computer? If you test that, make sure you 'pull' the files from another computer rather than 'push' them (eg originate the copy on the other computer). That should test your network and disk IO without involving Bacula. It would have to be an order of magnitude different at least to draw any real conclusions though. Then, after adding lowest-level (GZIP1) compression: Elapsed time: 2 mins 43 secs Priority: 10 FD Files Written: 4,648 SD Files Written: 4,648 FD Bytes Written: 308,509,378 (308.5 MB) SD Bytes Written: 309,228,685 (309.2 MB) Rate: 1892.7 KB/s Software Compression: 53.6 % VSS:yes Encryption: no Accurate: no Can you zip up the files to another computer? That might be a bit tricker to organise in a way that is similar to how bacula works... Any ideas what is the bottleneck here? I think 1GB of memory should be enough for XP alone. Some glue makes me think about the disk, but this rate is a decade below what to expect from even that old hard disk. The XP firewall at around sp2 was kind of broken in that the various tcp offloads didn't work properly and slowed things down to a crawl. Try disabling the firewall service in services.msc - just turning off the windows firewall on the network adapter is not sufficient, you have to stop the service. James -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
