Re: [Bacula-users] Windows client performance
Timo,. Your test involved just 600 some MB of data which may not be a large or varied enough data set. There is data and then there is data. Millions of small email index files is harder on Bacula than thousands of large files. And you don't mentioned if or how you controlled for your network bandwidth - did you keep all traffic off it? I'd suggest you use a data set which is typical of your intended use and multiply it by a factor of 10 and keep bandwidth stable. Mehma === -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
On 1/25/2010 11:07 AM, Timo Neuvonen wrote: some more cpu load, but no peak in workstation's cpu load meter exceeded 50%. In Windows, 100% load means all CPU's together at max load. If you have two cores, 50% means 100% load on one core. So you're seeing the best that CPU can do (and P4's are a seriously awful design), short of a major change in the compression library to utilize multiple threads. -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
Mike Ruskai than...@earthlink.net kirjoitti viestissä news:4b5dcf2f.9050...@earthlink.net... On 1/25/2010 11:07 AM, Timo Neuvonen wrote: some more cpu load, but no peak in workstation's cpu load meter exceeded 50%. In Windows, 100% load means all CPU's together at max load. If you have two cores, 50% means 100% load on one core. Actually, there were two graphs, one per core. Neither of them peaked close to 50% at GZIP1 level. After increasing comprerssion level (up to GZIP9), it needed more cpu load, but it wasn't absolutely the limiting factor if the graphs are somehow close to truth. But the basic question still is, what limits the data rate without compression, when cpu load is very low? Using compression still drops the compressed rate by same factor the data size shrinks, while the required real time doesn't practically chage at all. Sounds very much like it was a limit of disk io rate, but it sounds unbelievable to me since figures are that low. But maybe I'll need to test disk read performance first by some other means. It will just take a couple of days until I'll have time for that project again. So you're seeing the best that CPU can do (and P4's are a seriously awful design), short of a major change in the compression library to utilize multiple threads. -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
mehma sarja mehmasa...@gmail.com kirjoitti viestissä news:ec5d34681001250830h9cfb180naea17beb1a544...@mail.gmail.com... Timo,. Your test involved just 600 some MB of data which may not be a large or varied enough data set. There is data and then there is data. Millions of small email index files is harder on Bacula than thousands of large files. And you don't mentioned if or how you controlled for your network bandwidth - did you keep all traffic off it? I'd suggest you use a data set which is typical of your intended use and multiply it by a factor of 10 and keep bandwidth stable. Network load definetely wasn't the reason here, hard to document it, but I just know it. Also, the results were very repeatable when I run the same test several times, this doesn't sound like a network load problem either. But yes, I'll need to re-run the test with another type of data. A lot of small files will cause plenty of overhead. I just didn't think it would be that much overhead... -- Timo -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
On 1/25/2010 2:46 PM, Timo Neuvonen wrote: Mike Ruskaithan...@earthlink.net kirjoitti viestissä news:4b5dcf2f.9050...@earthlink.net... On 1/25/2010 11:07 AM, Timo Neuvonen wrote: some more cpu load, but no peak in workstation's cpu load meter exceeded 50%. In Windows, 100% load means all CPU's together at max load. If you have two cores, 50% means 100% load on one core. Actually, there were two graphs, one per core. Neither of them peaked close to 50% at GZIP1 level. After increasing comprerssion level (up to GZIP9), it needed more cpu load, but it wasn't absolutely the limiting factor if the graphs are somehow close to truth. You can't go by Task Manager graphs, either. Unless you set process affinity, the thread will be scheduled on alternating cores, and the graphs won't accurately represent how loaded each core is over time. If you have a single thread running at full speed, it will very likely look like roughly 50% usage on each core over time. The calculations just don't happen with a granularity small enough to show the actual usage. If you look at the CPU usage figure on the bottom left, however, that will be reasonably accurate. If it reads 50%, then the compression work is going as fast as it can. But the basic question still is, what limits the data rate without compression, when cpu load is very low? Using compression still drops the compressed rate by same factor the data size shrinks, while the required real time doesn't practically chage at all. Sounds very much like it was a limit of disk io rate, but it sounds unbelievable to me since figures are that low. But maybe I'll need to test disk read performance first by some other means. It will just take a couple of days until I'll have time for that project again. It could be either disc I/O or network I/O that's bottlenecking. The Bacula file daemon for Windows is definitely not restricted to the rates you're seeing. -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows client performance
Elapsed time: 2 mins 35 secs Priority: 10 FD Files Written: 4,648 SD Files Written: 4,648 FD Bytes Written: 664,739,011 (664.7 MB) SD Bytes Written: 665,458,318 (665.4 MB) Rate: 4288.6 KB/s Software Compression: None VSS:yes Encryption: no Accurate: no How long does it take to copy those files to a share on another windows computer? If you test that, make sure you 'pull' the files from another computer rather than 'push' them (eg originate the copy on the other computer). That should test your network and disk IO without involving Bacula. It would have to be an order of magnitude different at least to draw any real conclusions though. Then, after adding lowest-level (GZIP1) compression: Elapsed time: 2 mins 43 secs Priority: 10 FD Files Written: 4,648 SD Files Written: 4,648 FD Bytes Written: 308,509,378 (308.5 MB) SD Bytes Written: 309,228,685 (309.2 MB) Rate: 1892.7 KB/s Software Compression: 53.6 % VSS:yes Encryption: no Accurate: no Can you zip up the files to another computer? That might be a bit tricker to organise in a way that is similar to how bacula works... Any ideas what is the bottleneck here? I think 1GB of memory should be enough for XP alone. Some glue makes me think about the disk, but this rate is a decade below what to expect from even that old hard disk. The XP firewall at around sp2 was kind of broken in that the various tcp offloads didn't work properly and slowed things down to a crawl. Try disabling the firewall service in services.msc - just turning off the windows firewall on the network adapter is not sufficient, you have to stop the service. James -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
