[Bacula-users] performance problem - Windows TLS
Hi Kamil, 2 days ago I had got the same problem like you. Open client config file for windows and put Maximum Network Buffer Size = 65536 in FileDaemon :) It will resolve the problem Mariusz. +-- |This was sent by mariusz@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- EMC VNX: the world's simplest storage, starting under $10K The only unified storage solution that offers unified management Up to 160% more powerful than alternatives and 25% more efficient. Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] performance problem - Windows TLS
Hello After enabling TLS, I've noticed significant performance drawback. I've made some tests for both Linux (Fedora 13) and Windows XP clients. I've used 250MB tar archive. One file. No compression. BACKUP: Windows TLS 850 kB/s Windows NO_TLS 8500 kB/s Linux TLS 9274 kB/s Linux NO_TLS10819 kB/s RESTORE: WindowsTLS 5645 kB/s WindowsNO_TLS 8114 kB/s Linux TLS 5409 kB/s Linux NO_TLS 5901 kB/s As you can see, Windows backup with TLS enabled is 10 times slower than backup without TLS. On Linux clients I didn't notice this problem. I checked it few times, also with other PC. During backup client CPU load was around 1%. So I don't thing it's hardware problem. I've also copied tar archive from Windows client to backup storage server (using CIFS) - 16 MB/s. Am I missing somthing? Did you have simillar problems? What can cause this slowdown? Is it normal? How can I improve performance? Thanks in advance. Kamil +-- |This was sent by kamilfur...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] performance problem
FWIW the backups sped up considerably and finished after 1.5 days at an overall transfer rate of about 1.5 MB/s. I'm really not sure what caused the slowdown yesterday but the eventual speed up seems to imply an environmental state on the machines that went away. I checked to see if the AV software was doing full system scans, but it wasn't. Unless anyone has ideas on what might have been causing the slowdown, I will consider this issue (more or less) resolved. Thanks anyway. From: Jeff Shanholtz [mailto:jeffs...@shanholtz.com] Sent: Wednesday, August 03, 2011 5:31 PM To: Bacula-users@lists.sourceforge.net Subject: [Bacula-users] performance problem I currently have 3 clients doing a full backup (simultaneously). According to status client one is getting 300kb/s (this one is my director and storage server machine), one is getting 225kb/s, and one is getting 50kb/s. I've disabled AV on access scanning for the bacula-fd.exe process. I have software compression enabled, but none of the 3 systems seem bogged down so I think the bottleneck is not due to that option (although I'm tempted to turn on ntfs compression on the backup drive and disable software compression in the future). For the most part I don't mind too much that the backups are so slow because I'm quite happy to see the client machines continuing to be quite snappy. The main concern, particularly for full backups (which at this rate will take upwards of 3 days to complete), is the possibility of a system going offline and thus killing the backup (or will it pick up where it left off, as long as I have FD Connect Timeout configured to be longer than a system would typically be offline for, e.g. 12 hours?). So what else could be coming into play with my poor performance? Are simultaneous backups problematic performance-wise? I've watched the I/O activity of bacula-sd.exe and it certainly doesn't seem to be maxed out. It is using an external USB2 hard drive. Could the difference between USB2 and eSATA be the key? I can connect them as eSATA if I really need to. Seems like USB2 should be allowing substantially more than the roughly 600kb/s overall speed I'm getting though. I'm running all Windows Bacula binaries, version 3.0.3. I'm not sure posting config files is necessary at this point, although I'm happy to do so if needed. -- BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA The must-attend event for mobile developers. Connect with experts. Get tools for creating Super Apps. See the latest technologies. Sessions, hands-on labs, demos much more. Register early save! http://p.sf.net/sfu/rim-blackberry-1___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] performance problem
I currently have 3 clients doing a full backup (simultaneously). According to status client one is getting 300kb/s (this one is my director and storage server machine), one is getting 225kb/s, and one is getting 50kb/s. I've disabled AV on access scanning for the bacula-fd.exe process. I have software compression enabled, but none of the 3 systems seem bogged down so I think the bottleneck is not due to that option (although I'm tempted to turn on ntfs compression on the backup drive and disable software compression in the future). For the most part I don't mind too much that the backups are so slow because I'm quite happy to see the client machines continuing to be quite snappy. The main concern, particularly for full backups (which at this rate will take upwards of 3 days to complete), is the possibility of a system going offline and thus killing the backup (or will it pick up where it left off, as long as I have FD Connect Timeout configured to be longer than a system would typically be offline for, e.g. 12 hours?). So what else could be coming into play with my poor performance? Are simultaneous backups problematic performance-wise? I've watched the I/O activity of bacula-sd.exe and it certainly doesn't seem to be maxed out. It is using an external USB2 hard drive. Could the difference between USB2 and eSATA be the key? I can connect them as eSATA if I really need to. Seems like USB2 should be allowing substantially more than the roughly 600kb/s overall speed I'm getting though. I'm running all Windows Bacula binaries, version 3.0.3. I'm not sure posting config files is necessary at this point, although I'm happy to do so if needed. -- BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA The must-attend event for mobile developers. Connect with experts. Get tools for creating Super Apps. See the latest technologies. Sessions, hands-on labs, demos much more. Register early save! http://p.sf.net/sfu/rim-blackberry-1___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Performance problem on a Job of a filesystem with a lots of Files
I need to improve performance of a Job which backups 150 files (mail and File Server). I was compressing the files on disk in some tgz files first (tar and gzip) ,then backuping then on tape with Bacula, i was getting about: Job write elapsed time = 00:32:16, Transfer rate = 44.93 M Bytes/second Because disks space constraints i have to delete the compressed files on disk and backup direct to tape, then i am getting about: Job write elapsed time = 03:56:37, Transfer rate = 11.71 M Bytes/second I think i am getting worst performance because of ramdon disk access speed, is that true? The Tape (LTO4) and Bacula Storage Daemon and Director are on different server than the file and mail server i am backuping up. Could someone has some suggestions about improving speed from Bacula or from filesystem tuning? Does Bacula software compression might help on this? I Have a HP E200 SAS Disk Controller with 512MB (with Battery Write Cache) in a Raid 5 LVM (File and Mail Server). Thanks in advance for your advices, -- Andrés Fernando Yacopino -- Nokia and ATT present the 2010 Calling All Innovators-North America contest Create new apps games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Performance problem on a Job of a filesystem with a lots of Files
I need to improve performance of a Job which backups 150 files (mail and File Server). I was compressing the files on disk in some tgz files first (tar and gzip) ,then backuping then on tape with Bacula, i was getting about: Job write elapsed time = 00:32:16, Transfer rate = 44.93 M Bytes/second Because disks space constraints i have to delete the compressed files on disk and backup direct to tape, then i am getting about: Job write elapsed time = 03:56:37, Transfer rate = 11.71 M Bytes/second I think i am getting worst performance because of ramdon disk access speed, is that true? The Tape (LTO4) and Bacula Storage Daemon and Director are on different server than the file and mail server i am backuping up. Could someone has some suggestions about improving speed from Bacula or from filesystem tuning? Does Bacula software compression might help on this? Software compression will only slow things down. HW compression happens on the tape drive and the tape drive is capable of compressing at at lest 120MB/s. I Have a HP E200 SAS Disk Controller with 512MB (with Battery Write Cache) in a Raid 5 LVM (File and Mail Server). Do you have spooling on? Attribute spooling will greatly reduce the load on the database. Instead of adding an entry in the database for each file as the file is backed up the datbase is updated after the job completes. Data spooling can however slow down an individual job (since spooling and despooling are not concurrent) but it will speed up the situation where you have more than 1 job you want to run. There are many theories on what spool size to use, I use 5GB. So bacula reads 5GB of data to the spool then stops the reading from disk and writes that 5 GB to the tape. Since I have multiple clients I run several jobs concurrently so I get closer to the full bandwith out of my dual drive LTO2 archive. Each tape drive is writing at 30 to 45 MB/s for most of the time while my backups are in progress. John -- Nokia and ATT present the 2010 Calling All Innovators-North America contest Create new apps games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Performance problem on a Job of a filesystem with a lots of Files
On 23/09/10 15:26, Andrés Yacopino wrote: I think i am getting worst performance because of ramdon disk access speed, is that true? Yes. If you use the time command on your tar process you will find it is similarly slow. Actually it's not so much random disk access speed as the fixed time involved in stat() and open() on each file, no matter what its size is. Things are compounded when there are a lot of files in one directory. My experiments with GFS2 show that past 4000 files/directory the performance for ls and open operations deteriorate rapidly. Other filesystems have different thresholds but they _ALL_ perform poorly when a directory has too many files in it. 2 examples: 2 filesystems, both 1Tb, both 95% full One has 7,650 files in it. That copies to the spool disk at an average speed of 80Mb/s One has 3,500,000 files in it. That one only averages 15Mb/s to the spool disk. (Spool is all SSD and the FD-SD link is 1Gb/s. The limits are imposed by the -fd machine's filesystem and underlying disk arrays. If the filesystems are in use then things get much slower.) -- Nokia and ATT present the 2010 Calling All Innovators-North America contest Create new apps games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Performance Problem
I have bacula running (version 2.0.2) and in principle everything works = fine. But now (reading some mails from the list) I ask myself why the = backup-speed is that slow. In average it's about 1500 kb/s. Is this an incremental or Differential backup? John - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Performance Problem
Hi all! I have bacula running (version 2.0.2) and in principle everything works = fine. But now (reading some mails from the list) I ask myself why the = backup-speed is that slow. In average it's about 1500 kb/s. The software is running on fedora. The Computer has a fast CPU and 2GB = of RAM. No network backups, just lokal disk. The backup-drive is a lto-1 = hp. What backup-speed coult i expect? How could i find the bottleneck? Thank you for your assistance. Rainer No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.487 / Virus Database: 269.13.22/1013 - Release Date: 17.09.2007 13:29 - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Performance Problem
In response to Rainer Hackel [EMAIL PROTECTED]: Hi all! I have bacula running (version 2.0.2) and in principle everything works = fine. But now (reading some mails from the list) I ask myself why the = backup-speed is that slow. In average it's about 1500 kb/s. The software is running on fedora. The Computer has a fast CPU and 2GB = of RAM. No network backups, just lokal disk. The backup-drive is a lto-1 = hp. What backup-speed coult i expect? How could i find the bottleneck? How fast are your disks/tapes? Are you backing up to tape or disk? Try some dd tests to see how fast you can transfer data raw. Use tar going from disk to tape to see how fast that runs, and/or use dd going from disk to disk. Frequently, in my experience, otherwise fast computers have slow (but reliable) hard drives in them. Since RAM is so cheap, you usually don't notice this until you're moving _lots_ of data around. From there, you have to take into account that the DBMS has to write the catalog records, so you could run some tests to see how fast it can write new records to see if that's slowing you down. In my experience, CPU/memory are usually not the bottleneck when backups are running. -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Performance Problem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Rainer Hackel wrote: I have bacula running (version 2.0.2) and in principle everything works = fine. I feel obliged to warn you about that version: http://www.bacula.org/downloads/bug-395.txt You should upgrade to 2.2.4 as soon as possible. On a sidenote, 2.2.4 has some performance improvements in the way it inserts data into SQL (batch inserts) but will not work on very ancient MySQL versions for example. Greetings, Michel -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) iD8DBQFG7/wq2Vs+MkscAyURApRHAKCKwTlA/rSdYMow8zBSB9GN03TgtACfbHu6 DiqG4c+wOFWl4MwWm5sEsEQ= =1YrL -END PGP SIGNATURE- - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Performance Problem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 John Drescher wrote: I have bacula running (version 2.0.2) and in principle everything works = fine. But now (reading some mails from the list) I ask myself why the = backup-speed is that slow. In average it's about 1500 kb/s. We are using an HP StorageWorks Ultrium 215 (LTO1) here. Using btape's test, I usually get 7500KB/s. When backing up local disks or some of our faster boxes over the network, I usually get 10MB/s. David Blewett -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG7/geZmlc6wNjtLYRAlsOAJ92+2hmhZcngMs7sUo7/hDGw6LP2gCgrUST VOstFpQH3c5QhY8kF1P1nH0= =S5lc -END PGP SIGNATURE- - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] performance problem on windows
Hello, On Wed, 13 Jul 2005, Carsten Schurig wrote: The backup of the Linux servers runs with about 800 kBytes/s (DDS-3 tapes), but the Windows server just returns about 100 kB/s, which is much too slow to backup 15 GB! Have you tried spooling? http://www.bacula.org/rel-manual/Data_Spooling.html --- This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual core and dual graphics technology at this free one hour event hosted by HP, AMD, and NVIDIA. To register visit http://www.hp.com/go/dualwebinar ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] performance problem on windows
On Wednesday 13 July 2005 12:14, Jonas Björklund wrote: Hello, On Wed, 13 Jul 2005, Carsten Schurig wrote: The backup of the Linux servers runs with about 800 kBytes/s (DDS-3 tapes), but the Windows server just returns about 100 kB/s, which is much too slow to backup 15 GB! Have you tried spooling? http://www.bacula.org/rel-manual/Data_Spooling.html The problem is that the client system is not sending data fast enough, so I don't see how spooling will help. I also have this problem, some Windows machines can manange ~1MB/s. One laptop in particular running WindowsXP can do no better than 50KB/s when backing up. Which is obviously no good to anyone. How can we debug the performance problems of the Windows FD? --- This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual core and dual graphics technology at this free one hour event hosted by HP, AMD, and NVIDIA. To register visit http://www.hp.com/go/dualwebinar ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users Cheers, -- Dominic Marks --- This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual core and dual graphics technology at this free one hour event hosted by HP, AMD, and NVIDIA. To register visit http://www.hp.com/go/dualwebinar ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
