Re: [Bacula-users] Network error with FD during Backup: ERR=Connection reset by peer
There are Job options to reschedule jobs on error: Reschedule On Error = yes Reschedule Interval = 30 minutes Reschedule Times = 18 The above will reschedule the job 30 minutes after the failure and it'll try and do that 18 times before finally giving up. These options come in handy if you're backing up laptops or other computers that may not be on your network 24x7. hope this helps, I never have had to use these options, but while you mention it now, I remember, that I had read about them in the past. :) Thanks for the hint. I will try to use it, hopefully just as a workaround. Michael NovaNet GmbH Kupferstr. 65 44532 Lunen Telefon: 02306/202100 FAX: 02306/202109 WEB: www.novanetgmbh.de Firmensitz: Lunen Amtsgericht Dortmund HRB 17273 USt-ID DE 124793480, St.-Nr. 316/5759/0318 Geschaftsfuhrerin: Dipl. Informatikerin (FH) Desiree Wunsche -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extremely Slow Performance on VMware
Op 20120927 om 21:55 schreef Rodrigo Abrantes Antunes: Citando Davide Franco dfra...@dflc.ch: On Thu, Sep 27, 2012 at 7:04 PM, Rodrigo Abrantes Antunes wrote: Hi, I'm having extremely slow performance when backing up, a 10gb backup took around 3h. Where should I start looking? During a backup I can see bacula-sd using 100% cpu, memory usage is normal. My system is a vmware virtual machine with a cpu with two 2,5ghz cores and 2gb ram. The vm have a separate partition for the volumes using ext4 and mounted via RDM in a fc storage array with raid6. I tried mapping more cores and more ram for the vm with the same result so I don't think the problem is lack of resources. giving more informations like snip/ will give us more ideas to troubleshoot your problem. Director: 5.0.1-1ubuntu1 Storage: 5.0.1-1ubuntu1 FD: 5.0.1-1ubuntu1 (some clients have lower version) Database: mysqI OS: Ubuntu 10.04.4 x64 Server FC Storage 4 GBits/s. All my network is Gigabit Ethernet. Yes, and how is the further design? In others words: The provided list can read as One physical computer with fibre channel disk hosts all the VMs. If it is so, then tell so. Otherwise eloborate the setup, the design. One question: can vmware tools have any influence in this case? From what I know vmware tools is mainly for better graphics performance isn't it? I din't installed it. There could also be beter disk drivers. The virtio drivers come to mind. Back to During a backup I can see bacula-sd using 100% cpu, And where did you see the 100%? ( Which tool was used to read that performance valule? ) I would like to see the output of vmstat 2 5 during non-back-up-time and also the output of vmstat 2 5 during back-up-time. Thing I'm interrested in, are the CPU columns. Especial the colums system and wait. screenshot $ vmstat 2 3 procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si sobibo in cs us sy id wa 0 0 6712 11360 192584 15912000 5 46 11 5 6 90 0 0 0 6712 11344 192584 15912000 0 0 67 342 7 15 79 0 0 0 6712 11344 192584 15912000 0 0 65 340 8 14 78 0 /screenshot And to avoid an extra e-mail exchange: I'm asking for 2 to the power 3, so 8 measurements. So 2 moments (during backup or outside backup) on 2 Bacula compoments ( storage deamon and file deamon ) on the VMs on 2 physical hosts. Yes, that means that I assume the VMware hosts have a 'vmstat' command. That is because I'm not familair with VMware, I'm from the Xen world :-) Cheers Geert Stappers -- http://www.vanadcimplicity.com/ -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extremely Slow Performance
What kind of files are you backing up? I've seen speeds as low as 0,5 MB / sec on a more or less idle host which has a gazillion of small image files on it. In fact I've needed to split up the backup into several sub-jobs to prevent it from running more than 48h or so ;-) Also, I've seen jobs which start off incredibly slow only to pick up speed near the end... very weird. (bacula version 5.2.6 in this case). Have you checked your mysql tables for a corrupt index on the File or Filename tables? All the best, Uwe -- NIONEX --- Ein Unternehmen der Bertelsmann SE Co. KGaA -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extremely Slow Performance
Hello, 2012/9/27 Rodrigo Abrantes Antunes rodrigoantu...@pelotas.ifsul.edu.br Citando Davide Franco dfra...@dflc.ch: On Thu, Sep 27, 2012 at 7:04 PM, Rodrigo Abrantes Antunes rodrigoantu...@pelotas.ifsul.edu.br wrote: Hi, I'm having extremely slow performance when backing up, a 10gb backup took around 3h. It is about 1MB/s. Where should I start looking? In various places. :) During a backup I can see bacula-sd using 100% cpu, memory usage is normal. My system is a vmware virtual machine with a cpu with two 2,5ghz cores and 2gb ram. The vm have a separate partition for the volumes using ext4 and mounted via RDM in a fc storage array with raid6. I tried mapping more cores and more ram for the vm with the same result so I don't think the problem is lack of resources. Thanks. Director: 5.0.1-1ubuntu1 Storage: 5.0.1-1ubuntu1 It is fairly old version of Bacula. Is it possible to upgrade? FD: 5.0.1-1ubuntu1 (some clients have lower version) Database: mysqI OS: Ubuntu 10.04.4 x64 Server FC Storage 4 GBits/s. All my network is Gigabit Ethernet. First you have to check if your storage server is able to write data to disk fast enough. You can do a test with a simple dd if=/dev/zero of=/your-storage-filesystem bs=65536. Second you have to check if you set Spool Attributes = yes in your job, if not change it. You'll have to setup a spool directory for your device too. Next check a network speed. What is your RTT between client and storage daemon? I know it is a 1Gbit, but it is possible that you cannot utilize so much. You can test with a simple ftp client. Is it possible that you have a Firewall between both servers? Last you have to check a client utilization and filesystem read speed. How many small files you have to backup? Every new file to backup has its own overhead. Bacula has to find this file, stat it, open it, read data from it (with 64k block) and finally close it. If you have a 10M files of 1kB then your backup will be much slower then 1 file of 10GB. You have to be aware of it. One question: can vmware tools have any influence in this case? From what I know vmware tools is mainly for better graphics performance isn't it? I din't installed it. It is possible that without vmware tools you are using an emulated storage and network I/O instead of much faster virtualized. best regards -- Radosław Korzeniewski rados...@korzeniewski.net -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extremely Slow Performance
Hello, Is the compression enabled? I had this kind of problem because of the compression. Regards Gael Guilmin From: Rodrigo Abrantes Antunes [mailto:rodrigoantu...@pelotas.ifsul.edu.br] Sent: Thursday, September 27, 2012 9:55 PM To: Davide Franco Cc: bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Extremely Slow Performance Citando Davide Franco dfra...@dflc.chmailto:dfra...@dflc.ch: On Thu, Sep 27, 2012 at 7:04 PM, Rodrigo Abrantes Antunes rodrigoantu...@pelotas.ifsul.edu.brmailto:rodrigoantu...@pelotas.ifsul.edu.br wrote: Hi, I'm having extremely slow performance when backing up, a 10gb backup took around 3h. Where should I start looking? During a backup I can see bacula-sd using 100% cpu, memory usage is normal. My system is a vmware virtual machine with a cpu with two 2,5ghz cores and 2gb ram. The vm have a separate partition for the volumes using ext4 and mounted via RDM in a fc storage array with raid6. I tried mapping more cores and more ram for the vm with the same result so I don't think the problem is lack of resources. Thanks. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.netmailto:Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users Hi Rodrigo, giving more informations like * version of the director, storage, fd * which database back-end are you using * did you install the vmware tools ? * which operating system (version) are using ? * on what kind of storage does your DRM point to (sata, iscsi, fc, etc.) * etc. will give us more ideas to troubleshoot your problem. Regards Davide Director: 5.0.1-1ubuntu1 Storage: 5.0.1-1ubuntu1 FD: 5.0.1-1ubuntu1 (some clients have lower version) Database: mysqI OS: Ubuntu 10.04.4 x64 Server FC Storage 4 GBits/s. All my network is Gigabit Ethernet. One question: can vmware tools have any influence in this case? From what I know vmware tools is mainly for better graphics performance isn't it? I din't installed it. Thanks This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message. -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] file signatures PKI vs FileSet
Zitat von lst_ho...@kwsoft.de: Hello i wonder which of the cryptographic signatures is used and how if i specify in the FileSet option signature=md5 and on the client FD config PKI Signatures=yes. In the manual is stated that the PKI Signatures is not configurable but uses SHA-2 if available, otherwise SHA-1. This lead to the following questions: - Is the signature upgraded in this case to SHA-1? - Is one of them silently ignored if both are specified or will both be calculated and used? Further digging in to this one it looks like the signature configured in the FileSet is stored in the DB while PKI signature is part of the data, so both will be used/calculated. So it boils down to the following: - PKI Signature ensures that the client FD can verify on restore that the data are actually saved by itself signed with the private key - PKI Encryption ensures that no one without any of the private keys used at backup time can read the data - The FileSet signature (md5/sha) is used to compare (at Bacula SD?) if the data read are unaltered regarding the hash value stored in the database With this in mind we will switch off PKI signatures to eventuelly (re)gain some speed. Please let me know if i got something wrong on this topic Thanks Andreas -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] console messages
I don't log into bconsole for maybe a week at at time and when I do I'm usually messing about with config settings. I tend to always run 'auto on' as the first command but can sometimes have to wait over 5 minutes for a week's worth of log entries to scroll up my screen. Is there a way to limit the entries displayed to say the last 4 hours, or the last 1000 lines? -- Gary Stainburn Group I.T. Manager Ringways Garages http://www.ringways.co.uk -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] file signatures PKI vs FileSet
Hello, 2012/9/28 lst_ho...@kwsoft.de - PKI Signature ensures that the client FD can verify on restore that the data are actually saved by itself signed with the private key Correct. - PKI Encryption ensures that no one without any of the private keys used at backup time can read the data And Master Key if configured. :) - The FileSet signature (md5/sha) is used to compare (at Bacula SD?) if the data read are unaltered regarding the hash value stored in the database I think fileset signature is not used during restore. :) I can't find appropriate code in extract_data function. Maybe it is computed elsewhere. With this in mind we will switch off PKI signatures to eventuelly (re)gain some speed. It's a good idea. Especially that pki signatures are computed after whole file restore, which could be slow. best regards -- Radosław Korzeniewski rados...@korzeniewski.net -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] file signatures PKI vs FileSet
Zitat von Radosław Korzeniewski rados...@korzeniewski.net: Hello, 2012/9/28 lst_ho...@kwsoft.de - PKI Signature ensures that the client FD can verify on restore that the data are actually saved by itself signed with the private key Correct. - PKI Encryption ensures that no one without any of the private keys used at backup time can read the data And Master Key if configured. :) - The FileSet signature (md5/sha) is used to compare (at Bacula SD?) if the data read are unaltered regarding the hash value stored in the database I think fileset signature is not used during restore. :) I can't find appropriate code in extract_data function. Maybe it is computed elsewhere. As far as i know the fielset signature is used for verify jobs, so it would be the Storage Daemon using it and not the File Daemon, no? With this in mind we will switch off PKI signatures to eventuelly (re)gain some speed. It's a good idea. Especially that pki signatures are computed after whole file restore, which could be slow. Thanks for confirming Andreas -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO3 tape capacity (variable?)
On 28/09/12 02:38, Stephen Thompson wrote: Aren't these considered reasonable settings for LTO3? Maximum block size = 262144 # 256kb Maximum File Size = 2gb Not really. Change maximum file size to 10Gb and maximum block size to 2M You _must_ set all open tapes to used and restart the storage daemon when changing the block size. Bacula can't cope with varying maximum sizes on a tape Even with those changes, if you have a lot of small, incompressible files you'll see high tape overheads. thanks for the help! Stephen -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Unexpected cryptographic session data stream on restore
Hello
we like to use data encryption but get trouble on restore with the
following error:
28-Sep 13:30 bacula-dir JobId 100: Start Restore Job
RestoreFiles.2012-09-28_13.30.33_06
28-Sep 13:30 bacula-dir JobId 100: Using Device FileStorage
28-Sep 13:30 bacula-sd JobId 100: Ready to read from volume
file-0003 on device FileStorage (/var/lib/bacula/filestore).
28-Sep 13:30 bacula-sd JobId 100: Forward spacing Volume file-0003
to file:block 42:1538520297.
28-Sep 14:01 build-rst-fd JobId 100: Error: filed/restore.c:657
Unexpected cryptographic session data stream.
28-Sep 14:01 build-rst-fd JobId 100: Error: filed/restore.c:657
Unexpected cryptographic session data stream.
and so on
Most of the data in the same job restores fine, but after this error
we see also many files with 0 bytes size. The scenario is kept simple
to rule out operator errors. We have 3 windows machines (2x 64 bit xp,
1x 32 bit xp) and a Ubuntu Linux 12.04x64 as server. Clients running
Bacula windows client 5.2.10 32/64 bit and the server Bacula 5.2.12.
The relevant part of the config from client is
FileDaemon {# this is me
Name = buildx64-fd
FDport = 9102# where we listen for the director
WorkingDirectory = C:\\Program Files\\Bacula\\working
Pid Directory = C:\\Program Files\\Bacula\\working
Maximum Concurrent Jobs = 10
PKI Signatures = yes
PKI Encryption = yes
PKI Master Key = c:/program files/bacula/master-backup.cert
PKI Keypair = c:/program files/bacula/buildx64-backup.pem
}
We have tried two days with many saves and restores on the same
machine and on different machines with the same keys to no avail. Any
idea how to debug this further on. How can i get the filename for the
first error with the file:block 42:1538520297 info?
Thanks for any help
Regards
Andreas
--
Got visibility?
Most devs has no idea what their production app looks like.
Find out how fast your code is with AppDynamics Lite.
http://ad.doubleclick.net/clk;262219671;13503038;y?
http://info.appdynamics.com/FreeJavaPerformanceDownload.html
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Unexpected cryptographic session data stream on restore
Zitat von lst_ho...@kwsoft.de:
Hello
we like to use data encryption but get trouble on restore with the
following error:
28-Sep 13:30 bacula-dir JobId 100: Start Restore Job
RestoreFiles.2012-09-28_13.30.33_06
28-Sep 13:30 bacula-dir JobId 100: Using Device FileStorage
28-Sep 13:30 bacula-sd JobId 100: Ready to read from volume
file-0003 on device FileStorage (/var/lib/bacula/filestore).
28-Sep 13:30 bacula-sd JobId 100: Forward spacing Volume file-0003
to file:block 42:1538520297.
28-Sep 14:01 build-rst-fd JobId 100: Error: filed/restore.c:657
Unexpected cryptographic session data stream.
28-Sep 14:01 build-rst-fd JobId 100: Error: filed/restore.c:657
Unexpected cryptographic session data stream.
and so on
Most of the data in the same job restores fine, but after this error
we see also many files with 0 bytes size. The scenario is kept simple
to rule out operator errors. We have 3 windows machines (2x 64 bit xp,
1x 32 bit xp) and a Ubuntu Linux 12.04x64 as server. Clients running
Bacula windows client 5.2.10 32/64 bit and the server Bacula 5.2.12.
The relevant part of the config from client is
FileDaemon {# this is me
Name = buildx64-fd
FDport = 9102# where we listen for the director
WorkingDirectory = C:\\Program Files\\Bacula\\working
Pid Directory = C:\\Program Files\\Bacula\\working
Maximum Concurrent Jobs = 10
PKI Signatures = yes
PKI Encryption = yes
PKI Master Key = c:/program files/bacula/master-backup.cert
PKI Keypair = c:/program files/bacula/buildx64-backup.pem
}
We have tried two days with many saves and restores on the same
machine and on different machines with the same keys to no avail. Any
idea how to debug this further on. How can i get the filename for the
first error with the file:block 42:1538520297 info?
It looks like the error arises if the FD does not overwrite some files
on restore. This confuses the PKI code and subsequent file restore
fails also. In our case it was a fresh formated NTFS partition with a
restore with replace=never and the hidden System Volume Information
stops the restore.
There was also a report some time ago:
http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg50628.html
Should i create a bug report or have i missed something obvious?
Regards
Andreas
--
Got visibility?
Most devs has no idea what their production app looks like.
Find out how fast your code is with AppDynamics Lite.
http://ad.doubleclick.net/clk;262219671;13503038;y?
http://info.appdynamics.com/FreeJavaPerformanceDownload.html
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extremely Slow Performance on VMware
I have a physical machine that is a Vmware ESX node wich hosts only one vm, the one with bacula-director, bacula-sd and bacula-fd (called bacula-server), this vm has an RDM with the fibre channel storage where the volumes partition is mounted. Then I have all my clients (some are physycal machines and others are vms in other ESX nodes) with bacula-fd that are backed up. When I manually run a job to backup one of these clients in bacula-server I can see (with the command htop) that bacula-sd is using 100% of the cpu, I also noted that the backup starts at around 4MB/s and after some time it is around 300KB/s. If I simple send the same files to be backed up with scp for example the transfer goes around 100MB/s. The vms don't have vmstat, I use linux own commands. Citando Geert Stappers geert.stapp...@vanadgroup.com: Op 20120927 om 21:55 schreef Rodrigo Abrantes Antunes: Citando Davide Franco dfra...@dflc.ch: On Thu, Sep 27, 2012 at 7:04 PM, Rodrigo Abrantes Antunes wrote: Hi, I'm having extremely slow performance when backing up, a 10gb backup took around 3h. Where should I start looking? During a backup I can see bacula-sd using 100% cpu, memory usage is normal. My system is a vmware virtual machine with a cpu with two 2,5ghz cores and 2gb ram. The vm have a separate partition for the volumes using ext4 and mounted via RDM in a fc storage array with raid6. I tried mapping more cores and more ram for the vm with the same result so I don't think the problem is lack of resources. giving more informations like snip/ will give us more ideas to troubleshoot your problem. Director: 5.0.1-1ubuntu1 Storage: 5.0.1-1ubuntu1 FD: 5.0.1-1ubuntu1 (some clients have lower version) Database: mysqI OS: Ubuntu 10.04.4 x64 Server FC Storage 4 GBits/s. All my network is Gigabit Ethernet. Yes, and how is the further design? In others words: The provided list can read as One physical computer with fibre channel disk hosts all the VMs. If it is so, then tell so. Otherwise eloborate the setup, the design. One question: can vmware tools have any influence in this case? From what I know vmware tools is mainly for better graphics performance isn't it? I din't installed it. There could also be beter disk drivers. The virtio drivers come to mind. Back to During a backup I can see bacula-sd using 100% cpu, And where did you see the 100%? ( Which tool was used to read that performance valule? ) I would like to see the output of vmstat 2 5 during non-back-up-time and also the output of vmstat 2 5 during back-up-time. Thing I'm interrested in, are the CPU columns. Especial the colums system and wait. screenshot $ vmstat 2 3 procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 6712 11360 192584 159120 0 0 5 4 6 11 5 6 90 0 0 0 6712 11344 192584 159120 0 0 0 0 67 342 7 15 79 0 0 0 6712 11344 192584 159120 0 0 0 0 65 340 8 14 78 0 /screenshot And to avoid an extra e-mail exchange: I'm asking for 2 to the power 3, so 8 measurements. So 2 moments (during backup or outside backup) on 2 Bacula compoments ( storage deamon and file deamon ) on the VMs on 2 physical hosts. Yes, that means that I assume the VMware hosts have a 'vmstat' command. That is because I'm not familair with VMware, I'm from the Xen world :-) Cheers Geert Stappers -- http://www.vanadcimplicity.com/ -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extremely Slow Performance
Citando Uwe Schuerkamp uwe.schuerk...@nionex.net: What kind of files are you backing up? I've seen speeds as low as 0,5 MB / sec on a more or less idle host which has a gazillion of small image files on it. In fact I've needed to split up the backup into several sub-jobs to prevent it from running more than 48h or so ;-) Also, I've seen jobs which start off incredibly slow only to pick up speed near the end... very weird. (bacula version 5.2.6 in this case). Have you checked your mysql tables for a corrupt index on the File or Filename tables? All the best, Uwe --NIONEX --- Ein Unternehmen der Bertelsmann SE Co. KGaA I'm backing up multiple different clients, some with lots of small files like the mail server and others with big files. All of them starts the backup fast and end slow. Here is the show index, I don't know if it's right or not: show index from File; +---++--+--+-+---+-+--++--++-+ | File | 0 | PRIMARY | 1 | FileId | A | 274094 | NULL | NULL | | BTREE | | | File | 1 | JobId | 1 | JobId | A | 31 | NULL | NULL | | BTREE | | | File | 1 | JobId_2 | 1 | JobId | A | 31 | NULL | NULL | | BTREE | | | File | 1 | JobId_2 | 2 | PathId | A | 3559 | NULL | NULL | | BTREE | | | File | � � 1 | JobId_2 | 3 | FilenameId | A | 274094 | NULL | NULL | | BTREE | | +---++--+--+-+---+-+--++--++-+ show index from Filename; +--++--+--+-+---+-+--++--++-+ | Filename | 0 | PRIMARY | 1 | FilenameId | A | 1588498 | NULL | NULL | | BTREE | | | Filename | 1 | Name | 1 | Name | A | 1588498 | 255 | NULL | | BTREE | | +--++--+--+-+---+-+-- ++--++-+ show index from Path; +---++--+--+-+---+-+--++--++-+ | Path | 0 | PRIMARY | 1 | PathId | A | 40618 | NULL | NULL | | BTREE | | | Path | 1 | Path | 1 | Path | A | 40618 | 255 | NULL | | BTREE | | +---++--+--+-+---+-+--++--++-+ -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] HOWTO - FileSet from MD5 to SHA1
Hi All, I want to change the File Signature type encryption from MD5 to SHA1 on one of my client FileSet. Will changing it from MD5 to SHA1 trigger a Full Backup ? Whats the correct way of doing this? Should i just change the FileSet encryption from MD5 to SHA1 and restart bacula-dir and all will be well ?? Thanks in advance attachment: hbryant.vcf-- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Unexpected cryptographic session data stream on restore
On Fri, 28 Sep 2012 17:25:42 +0200, lst hoe02 said: It looks like the error arises if the FD does not overwrite some files on restore. This confuses the PKI code and subsequent file restore fails also. In our case it was a fresh formated NTFS partition with a restore with replace=never and the hidden System Volume Information stops the restore. There was also a report some time ago: http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg50628.html Should i create a bug report or have i missed something obvious? I think you should report it as a bug. __Martin -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extremely Slow Performance
It's the version that is in ubuntu 10.04 repos and I can't update the system right now. My storage array with raid6 can write up to 400MB/s and trough the network up to 100MB/s (Gigabit), these speeds were tested and confirmed and I don't use firewall between bacula-server and clients. I din't set spool attributes, can it increase speed? How? Sorry I'm new to bacula. f I simple send the same files to be backed up with scp for example the transfer goes around 100MB/s, so I don't think I'm using an emulated storage and network I/O. Citando Radosław Korzeniewski rados...@korzeniewski.net: Hello, 2012/9/27 Rodrigo Abrantes Antunes rodrigoantu...@pelotas.ifsul.edu.br Citando Davide Franco dfra...@dflc.ch: On Thu, Sep 27, 2012 at 7:04 PM, Rodrigo Abrantes Antunes rodrigoantu...@pelotas.ifsul.edu.br wrote: Hi, I'm having extremely slow performance when backing up, a 10gb backup took around 3h. It is about 1MB/s. Where should I start looking? In various places. :) During a backup I can see bacula-sd using 100% cpu, memory usage is normal. My system is a vmware virtual machine with a cpu with two 2,5ghz cores and 2gb ram. The vm have a separate partition for the volumes using ext4 and mounted via RDM in a fc storage array with raid6. I tried mapping more cores and more ram for the vm with the same result so I don't think the problem is lack of resources. Thanks. Director: 5.0.1-1ubuntu1 Storage: 5.0.1-1ubuntu1 It is fairly old version of Bacula. Is it possible to upgrade? FD: 5.0.1-1ubuntu1 (some clients have lower version) Database: mysqI OS: Ubuntu 10.04.4 x64 Server FC Storage 4 GBits/s. All my network is Gigabit Ethernet. First you have to check if your storage server is able to write data to disk fast enough. You can do a test with a simple dd if=/dev/zero of=/your-storage-filesystem bs=65536. Second you have to check if you set Spool Attributes = yes in your job, if not change it. You'll have to setup a spool directory for your device too. Next check a network speed. What is your RTT between client and storage daemon? I know it is a 1Gbit, but it is possible that you cannot utilize so much. You can test with a simple ftp client. Is it possible that you have a Firewall between both servers? Last you have to check a client utilization and filesystem read speed. How many small files you have to backup? Every new file to backup has its own overhead. Bacula has to find this file, stat it, open it, read data from it (with 64k block) and finally close it. If you have a 10M files of 1kB then your backup will be much slower then 1 file of 10GB. You have to be aware of it. One question: can vmware tools have any influence in this case? From what I know vmware tools is mainly for better graphics performance isn't it? I din't installed it. It is possible that without vmware tools you are using an emulated storage and network I/O instead of much faster virtualized. best regards -- Radosław Korzeniewski rados...@korzeniewski.net -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extremely Slow Performance
No compression is enabled. Citando Gael Guilmin gael.guil...@pdgm.com: Hello, Is the compression enabled? I had this kind of problem because of the compression. Regards Gael Guilmin FROM:Rodrigo Abrantes Antunes [mailto:rodrigoantu...@pelotas.ifsul.edu.br] SENT: Thursday, September 27, 2012 9:55 PM TO: Davide Franco CC: bacula-users@lists.sourceforge.net SUBJECT: Re: [Bacula-users] Extremely Slow Performance Citando Davide Franco dfra...@dflc.ch: On Thu, Sep 27, 2012 at 7:04 PM, Rodrigo Abrantes Antunes rodrigoantu...@pelotas.ifsul.edu.br wrote: Hi, I'm having extremely slow performance when backing up, a 10gb backup took around 3h. Where should I start looking? During a backup I can see bacula-sd using 100% cpu, memory usage is normal. My system is a vmware virtual machine with a cpu with two 2,5ghz cores and 2gb ram. The vm have a separate partition for the volumes using ext4 and mounted via RDM in a fc storage array with raid6. I tried mapping more cores and more ram for the vm with the same result so I don't think the problem is lack of resources. Thanks. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users Hi Rodrigo, giving more informations like * version of the director, storage, fd * which database back-end are you using * did you install the vmware tools ? * which operating system (version) are using ? * on what kind of storage does your DRM point to (sata, iscsi, fc, etc.) * etc. will give us more ideas to troubleshoot your problem. Regards Davide Director: 5.0.1-1ubuntu1 Storage: 5.0.1-1ubuntu1 FD: 5.0.1-1ubuntu1 (some clients have lower version) Database: mysqI OS: Ubuntu 10.04.4 x64 Server FC Storage 4 GBits/s. All my network is Gigabit Ethernet. One question: can vmware tools have any influence in this case? From what I know vmware tools is mainly for better graphics performance isn't it? I din't installed it. Thanks This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message. -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] HOWTO - FileSet from MD5 to SHA1
On Fri, 28 Sep 2012 11:20:24 -0500 Humphrey Bryant hbry...@fogadaley.com wrote: Hi All, I want to change the File Signature type encryption from MD5 to SHA1 on one of my client FileSet. Will changing it from MD5 to SHA1 trigger a Full Backup ? Whats the correct way of doing this? Should i just change the FileSet encryption from MD5 to SHA1 and restart bacula-dir and all will be well ?? Thanks in advance I did the same thing. It does not require a new Full backup. -- Craig Van Tassle HUBzero.org System Administrator YONG 1006 Desk Phone : (765)496-6413 -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extremely Slow Performance on VMware
Citando Rodrigo Abrantes Antunes rodrigoantu...@pelotas.ifsul.edu.br: I have a physical machine that is a Vmware ESX node wich hosts only one vm, the one with bacula-director, bacula-sd and bacula-fd (called bacula-server), this vm has an RDM with the fibre channel storage where the volumes partition is mounted. Then I have all my clients (some are physycal machines and others are vms in other ESX nodes) with bacula-fd that are backed up. When I manually run a job to backup one of these clients in bacula-server I can see (with the command htop) that bacula-sd is using 100% of the cpu, I also noted that the backup starts at around 4MB/s and after some time it is around 300KB/s. If I simple send the same files to be backed up with scp for example the transfer goes around 100MB/s. The vms don't have vmstat, I use linux own commands. Citando Geert Stappers geert.stapp...@vanadgroup.com: Op 20120927 om 21:55 schreef Rodrigo Abrantes Antunes: Citando Davide Franco dfra...@dflc.ch: On Thu, Sep 27, 2012 at 7:04 PM, Rodrigo Abrantes Antunes wrote: Hi, I'm having extremely slow performance when backing up, a 10gb backup took around 3h. Where should I start looking? During a backup I can see bacula-sd using 100% cpu, memory usage is normal. My system is a vmware virtual machine with a cpu with two 2,5ghz cores and 2gb ram. The vm have a separate partition for the volumes using ext4 and mounted via RDM in a fc storage array with raid6. I tried mapping more cores and more ram for the vm with the same result so I don't think the problem is lack of resources. giving more informations like snip/ will give us more ideas to troubleshoot your problem. Director: 5.0.1-1ubuntu1 Storage: 5.0.1-1ubuntu1 FD: 5.0.1-1ubuntu1 (some clients have lower version) Database: mysqI OS: Ubuntu 10.04.4 x64 Server FC Storage 4 GBits/s. All my network is Gigabit Ethernet. Yes, and how is the further design? In others words: The provided list can read as One physical computer with fibre channel disk hosts all the VMs. If it is so, then tell so. Otherwise eloborate the setup, the design. One question: can vmware tools have any influence in this case? From what I know vmware tools is mainly for better graphics performance isn't it? I din't installed it. There could also be beter disk drivers. The virtio drivers come to mind. Back to During a backup I can see bacula-sd using 100% cpu, And where did you see the 100%? ( Which tool was used to read that performance valule? ) I would like to see the output of vmstat 2 5 during non-back-up-time and also the output of vmstat 2 5 during back-up-time. Thing I'm interrested in, are the CPU columns. Especial the colums system and wait. screenshot $ vmstat 2 3 procs ---memory-- ---swap-- -io -system-- cpu r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 6712 11360 192584 159120 0 0 5 4 6 11 5 6 90 0 0 0 6712 11344 192584 159120 0 0 0 0 67 342 7 15 79 0 0 0 6712 11344 192584 159120 0 0 0 0 65 340 8 14 78 0 /screenshot And to avoid an extra e-mail exchange: I'm asking for 2 to the power 3, so 8 measurements. So 2 moments (during backup or outside backup) on 2 Bacula compoments ( storage deamon and file deamon ) on the VMs on 2 physical hosts. Yes, that means that I assume the VMware hosts have a 'vmstat' command. That is because I'm not familair with VMware, I'm from the Xen world :-) Cheers Geert Stappers -- http://www.vanadcimplicity.com/ -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bacula-users mailing list Bacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users One thing I noted now, in the vm htop says that 100% cpu is used and actually the machine is very slow when backing up so I think this value is accurate but in VSphere Client in the performance chart it says that the vm is using only 400Mhz of the 5000Mhz that were allocated, but the node cpu usage is low so I don't know why it isn't aloccating more MHZ to the bacula-server. I installed vmstat, I can't do vmstat during non backup time because it is currently backing up my mail server, about 200gb, it is doing this for almost 15h: vmstat 2 5 during backing-up on the bacula-server (director, storagedaemon)
Re: [Bacula-users] newbie - problem labeling / adding tapes to pool
OK Hopefully I'm not annoying anyone with my updates. Here is the latest: I deleted the 5.2.6 install and installed 5.2.12, like so: backup the config files etc. delete all tapes by rewind, writing eof, rewind each one do a: make uninstall of bacula make sure that everything is deleted. I did this, and then deleted and recreated the bacula db, so Everything is wiped download the latest version and install, configure, used my previous config files see how bacula recognizes tapes And after doing an : barcode label, I'm basically back where I was. Only 4 out of 8 tapes are recognized. At least it's behaving consistently broken. *status slots The defined Storage resources are: 1: File 2: HP Autoloader Select Storage resource (1-2): 2 Connecting to Storage daemon HP Autoloader at pmappp01.tpmcomm.com:9103 ... 3306 Issuing autochanger slots command. Device HP-0 has 8 slots. Connecting to Storage daemon HP Autoloader at pmappp01.tpmcomm.com:9103 ... 3306 Issuing autochanger list command. Slot | Volume Name| Status | Media Type | Pool | --+--+---+--+| 1*| | ? |? | ? | 2*| | ? |? | ? | 3 | TPMBU001 |Append |LTO-4 | Default | 4 | TPMBU005 |Append |LTO-4 | Default | 5 | TPMBU007 |Append |LTO-4 | Default | 6 | TPMBU015 |Append |LTO-4 | Default | 7*| | ? |? | ? | 8*| | ? |? | ? | So at this point, I'm not sure what to think. When I do an: update slots scan, it returns the correct slot and correct Volume name for all tapes in all slots. I did this before doing a label barcodes, so it seems to be reading the barcodes (this is after writing eof) but when I do a label barcodes, it only labels 4 out of 8 tapes. It's like it only sees 4 slots during this operation. I'm going to see how backups run over the weekend, hopefully I'll get a full backup. I'm wondering if it's possible that a cabling issue could cause this, or some incompatibility with the controller card. Or 4 tapes went bad at the same time, but that seems extremely unlikely. I tried the btape test again, everything passed, even when using one of the 'mystery tapes' so I don't think it's the tape. When I check the mysql db, it says: mysql select concat(VolumeName,', ',Slot) from Media; +--+ | concat(VolumeName,', ',Slot) | +--+ | TPMBU001, 3 | | TPMBU005, 4 | | TPMBU007, 5 | | TPMBU015, 6 | +--+ 4 rows in set (0.00 sec) I'm sure people have had longer battles with bacula, but I'm getting discouraged. I'm thinking about taking the 4 good tapes out, connecting the tape drive to the old backup server, and trying backups using the same bad tapes and slots. That would rule out tapes and slots. any suggestions much appreciated, bacula is breaking me At least it's Friday, -- Got visibility? Most devs has no idea what their production app looks like. Find out how fast your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219671;13503038;y? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] newbie - problem labeling / adding tapes to pool
Hey Simon, The output from your status command is not returning the label information for the slots. Do you have another application that you can use to see if the autoloader is reading the barcode labels correctly? The following is what I get if there are no tapes in some slots (extra information is coming from BAT): status slots storage=Tape Connecting to Storage daemon Tape at backup:9103 ... 3306 Issuing autochanger listall command. D|0|15|000111L5 S|1|1|000112L5|191688698880|Append|LTO5|Archive|1346372575|4499972575 S|2|2|000202L5|68536581120|Purged|LTO5|Tape|1348009726|1348614526 S|3|3|000209L5|64512|Append|LTO5|Scratch|0|31536000 S|4|4|000210L5|64512|Append|LTO5|Scratch|0|31536000 S|5|5|000211L5|64512|Append|LTO5|Scratch|0|31536000 S|6|6|000212L5|64512|Append|LTO5|Scratch|0|31536000 S|7|7|000213L5|64512|Append|LTO5|Scratch|0|31536000 S|8 S|9|9|000214L5|64512|Append|LTO5|Scratch|0|31536000 S|10|10|000215L5|64512|Append|LTO5|Scratch|0|31536 S|11|11|000216L5|64512|Append|LTO5|Scratch|0|31536 S|12|12|000217L5|64512|Append|LTO5|Scratch|0|31536 S|13|13|000218L5|64512|Append|LTO5|Scratch|0|31536 S|14 S|15|15|000111L5|963895145472|Append|LTO5|Tape|1348849630|1349454430 S|16|16|CLNU00L1|1|Cleaning||Tape|0|31536 (slots 8 14 are empty) If I have new tapes in these slots that have not been labeled by bacula(but have barcode labels) I get the following : status slots storage=Tape Connecting to Storage daemon Tape at backup:9103 ... 3306 Issuing autochanger listall command. D|0|15|000111L5 S|1|1|000112L5|191688698880|Append|LTO5|Archive|1346372575|4499972575 S|2|2|000202L5|68536581120|Purged|LTO5|Tape|1348009726|1348614526 S|3|3|000209L5|64512|Append|LTO5|Scratch|0|31536000 S|4|4|000210L5|64512|Append|LTO5|Scratch|0|31536000 S|5|5|000211L5|64512|Append|LTO5|Scratch|0|31536000 S|6|6|000212L5|64512|Append|LTO5|Scratch|0|31536000 S|7|7|000213L5|64512|Append|LTO5|Scratch|0|31536000 S|8|0|000219L5|?|?|?|?|0|0 S|9|9|000214L5|64512|Append|LTO5|Scratch|0|31536000 S|10|10|000215L5|64512|Append|LTO5|Scratch|0|31536 S|11|11|000216L5|64512|Append|LTO5|Scratch|0|31536 S|12|12|000217L5|64512|Append|LTO5|Scratch|0|31536 S|13|13|000218L5|64512|Append|LTO5|Scratch|0|31536 S|14|0|000220L5|?|?|?|?|0|0 S|15|15|000111L5|963895145472|Append|LTO5|Tape|1348849630|1349454430 S|16|16|CLNU00L1|1|Cleaning||Tape|0|31536 Slot 8 14 show the barcode labels for the tapes, but no other information because they have not been labeled by bacula yet. From: Simon Tyler [mailto:si...@tpmcomm.com] Sent: Friday, September 28, 2012 3:43 PM To: Dan Langille Cc: bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] newbie - problem labeling / adding tapes to pool OK Hopefully I'm not annoying anyone with my updates. Here is the latest: I deleted the 5.2.6 install and installed 5.2.12, like so: backup the config files etc. delete all tapes by rewind, writing eof, rewind each one do a: make uninstall of bacula make sure that everything is deleted. I did this, and then deleted and recreated the bacula db, so Everything is wiped download the latest version and install, configure, used my previous config files see how bacula recognizes tapes And after doing an : barcode label, I'm basically back where I was. Only 4 out of 8 tapes are recognized. At least it's behaving consistently broken. *status slots The defined Storage resources are: 1: File 2: HP Autoloader Select Storage resource (1-2): 2 Connecting to Storage daemon HP Autoloader at pmappp01.tpmcomm.com:9103http://pmappp01.tpmcomm.com:9103 ... 3306 Issuing autochanger slots command. Device HP-0 has 8 slots. Connecting to Storage daemon HP Autoloader at pmappp01.tpmcomm.com:9103http://pmappp01.tpmcomm.com:9103 ... 3306 Issuing autochanger list command. Slot | Volume Name| Status | Media Type | Pool | --+--+---+--+| 1*| | ? |? | ? | 2*| | ? |? | ? | 3 | TPMBU001 |Append |LTO-4 | Default | 4 | TPMBU005 |Append |LTO-4 | Default | 5 | TPMBU007 |Append |LTO-4 | Default | 6 | TPMBU015 |Append |LTO-4 | Default | 7*| | ? |? | ? | 8*| | ? |? | ? | So at this point, I'm not sure what to think. When I do an: update slots scan, it returns the correct slot and correct Volume name for all tapes in all slots. I did this before doing a label barcodes, so it seems to be reading the barcodes (this is after writing eof) but when I do a label barcodes, it only labels 4 out of 8
