[Bacula-users] Recycling issue
Hello, I'm facing an issue with Bacula (5.2.12) on a Ubuntu server (12.04.4 LTS): I don't know why, but the server, after finishing writing on a tape, tries to use a tape that is not in the autochanger, despite the fact that there are plenty of online and usable tape in the Scratch pool... Any idea? Best regards, Gael -- System Administrator / IT [PDM_Logo_RGB - registered] 78, Avenue du XXeme Corps 54000 Nancy France Phone(Paris): +33 (0)1 49 49 05 25 Phone(Nancy): +33 (0)3 83 67 66 39 Cell: +33 (0)6 10 18 48 43 Web: www.pdgm.comhttp://www.pdgm.com/ [Description: Description: Description: http://www.pdgm.com/_media/newsletters/USA/USA_Edition3_June11_files/linkedin.jpg]http://www.linkedin.com/company/paradigm_2 [Description: Description: Description: http://www.pdgm.com/_media/newsletters/USA/USA_Edition3_June11_files/twitter.jpg] http://twitter.com/paradigmltd [Description: Description: Description: http://www.pdgm.com/_media/newsletters/USA/USA_Edition3_June11_files/facebook.jpg] http://www.facebook.com/pdgmltd [Description: Description: Description: http://www.pdgm.com/_media/newsletters/USA/USA_Edition3_June11_files/youtube.jpg] http://www.youtube.com/paradigmltd --- This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Error when trying to configure bacula-gui-7.0.5
Hi Guys Thank you for the response on this matter. I have now followed the instructions for installing baculum as per the INSTALL file, however now I a getting the following when I go to the web interface: The requested URL /wizard/ was not found on this server. I have done some web searches and everyone is noting that the authorisation needs to be configured and that mod_rewrite needs to be enabled. Both these have been done, however still getting that message. Any ideas on what might be wrong or how to fix this? I am by no means an apache or php expert. Regards On 31/07/2014 17:52, Marcin Haba wrote: 2014-07-31 15:05 GMT+02:00 Machiel mach...@integrat.co.za: Hi Guys I have installed bacula7.0.5 from source as well as bacula-web today without any issues. I am however trying to install bacula-gui-7.0.5 , but when I run the configure script I get the following: ./configure --with-bacula=/installs/bacula-7.0.5/ configuring for Bacula source 7.0.5 01Jan13 29Feb12 4.8.4 (28 July 2014) checking for true... /bin/true checking for false... /bin/false checking for gawk... gawk checking for gawk... /usr/bin/gawk checking for a BSD-compatible install... /usr/bin/install -c checking for mv... /bin/mv checking for rm... /bin/rm checking for cp... /bin/cp checking for echo... /bin/echo checking for ar... /usr/bin/ar checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking for Operating System Distribution... done configure: creating ./config.status config.status: creating autoconf/Make.common sed: file ./confstatvUTcN2/subs-1.sed line 35: unterminated `s' command config.status: creating Makefile sed: file ./confstatvUTcN2/subs-1.sed line 35: unterminated `s' command config.status: creating bimagemgr/Makefile config.status: error: cannot find input file: bimagemgr/Makefile.in I tried the same on another machine running 7.0.4 and got the exact same thing. (thought it might be a big with the latest version). I am currently running ubuntu 14.04 LTS server and need to get this up and running soon. Any help would be greatly appreciated. Hello, In bacula-gui archive there is one webGUI tool named Baculum. Script ../configure is not used for Baculum installation. For installing Baculum please look in next file: bacula-gui-7.0.5/baculum/INSTALL There is description about how to install Baculum. Best regards. Marcin Haba -- -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Disk based backup using vchanger, volumes being marked as Error
On 8/5/2014 1:36 AM, Kern Sibbald wrote: Hello Josh, Please see below ... On 08/04/2014 06:43 PM, Josh Fisher wrote: On 8/1/2014 12:27 PM, Joseph Dickson wrote: Greetings :-) I've run into this problem with Bacula in a previous installation, and I can't seem to recall if there was ever a resolution.. I'm using Bacula for disk based backups only, and I am using vchanger to manage my virtual library. I've configured a vchanger library with 100 slots and 8 drives, and have set a Maximum Volume Bytes of 100G on the pool definition that I am using, to limit each slot in the library to 100G. I have also set a Maximum Concurrent Jobs = 2 setting on each of the virtual tape drive devices in my storage director config, so that only two jobs can write to a device at a time to minimize interleaving. Everything works perfectly as long as I only kick a few jobs off at a time.. however, when my main backup windows run and 30 or 40 backup jobs kick off, I often end up with jobs that output the following sequence in the logs: Have you set PreferMountedVolumes=no in the Job resource in bacula-dir.conf? If 3 jobs start and want to write to volumes in the same pool, then all three can be assigned the same volume. In fact, if PreferMountedVolumes=yes, (the default), then all three WILL be assigned the same volume unless the pool restricts the max number of jobs that the volume may contain. However, your device (drive) restricts the max concurrent jobs to 2. Therefore one of those three jobs will not be able to select the drive where the volume is mounted and will be forced to select another unused drive. That third job will nevertheless select the same volume as the other two and attempt to move the volume from the drive it is in into the drive that it has been assigned to. The configuration has a built-in race condition. I have recently done quite a bit of work to try to avoid race conditions such as the one you describe above. Does this still happen on version 7.0.x? I ask because there is now code that *should* detect this and explicitly makes the third job (as you describe above) wait. Now it is possible that there is some code path in the SD where the new code does not apply, so I cannot exclude problems, but if any exist in 7.0.x I would like to know so I can work on it some more. With the new code, the Volume will be moved around, but at least it should be done correctly without some deadlock or failure. I haven't had a chance to update to 7.0.x yet, so I can't say. My thought is that the volume itself should have a "Maximum Concurrent Jobs" setting, in addition to the SD Device. Better still, it could be automated by forcing the volume's max concurrency to that of the SD device at mount time. That should eliminate the need for "Prefer Mounted Volumes" altogether, since once the "Maximum Concurrent Jobs" have selected the volume, subsequent jobs would reject it as unavailable and so see the drive it is mounted in as unavailable at drive selection time. Once a drive is selected, that volume would be viewed as unavailable and rejected during volume selection, at least until one of the jobs using the volume ends. So by setting "Max Concurrent Jobs" to 1, one could guarantee a volume would never be selected by more than one job at a time. Best regards, Kern Setting PreferMountedVolumes=no causes the three jobs to select a drive that is NOT already mounted with a volume from the pool. This allows jobs writing to the same pool to select different volumes from the pool, rather than all selecting the same next available volume. This has its own caveats. It doesn't necessarily prevent two jobs from selecting the same volume in some cases, meaning that they will want to swap the volume back and forth between drives, which is another type of race condition. I have used this method successfully for a pool containing full backups only by setting PreferMountedVolumes=no in the job resource and setting MaximumVolumeJobs=1 in the pool resource. Since Bacula selects the volume for a job in an atomic manner, this forces an exclusive set of volumes for each job, thus preventing the race
Re: [Bacula-users] Error when trying to configure bacula-gui-7.0.5
Hello, It looks that your DocumentRoot option from VirtualHost does not indicate to directory with Baculum files. The most probably is that you used subdirectory for storing Baculum files. For example, if you are storing Baculum files in location: /some/dir/bacula-gui/baculum then according Apache configuration file part can look like below: VirtualHost *:80 DocumentRoot /some/dir/bacula-gui/baculum ServerName somehost.example.com Directory /some/dir/bacula-gui/baculum AllowOverride All AuthType Basic AuthName MyPrivateFile AuthUserFile /etc/apache2/some_password_file Require valid-user /Directory /VirtualHost In other words, Baculum files need to be located on top location of your web server document root. From this reason access to Baculum is by: http://yourhost/ but NOT by: http://yourhost/baculum/ I believe that this problem is easy to skip by modifing mod_rewrite rules. Best regards. Marcin 2014-08-05 10:30 GMT+02:00 Machiel mach...@integrat.co.za: Hi Guys Thank you for the response on this matter. I have now followed the instructions for installing baculum as per the INSTALL file, however now I a getting the following when I go to the web interface: The requested URL /wizard/ was not found on this server. I have done some web searches and everyone is noting that the authorisation needs to be configured and that mod_rewrite needs to be enabled. Both these have been done, however still getting that message. Any ideas on what might be wrong or how to fix this? I am by no means an apache or php expert. Regards On 31/07/2014 17:52, Marcin Haba wrote: 2014-07-31 15:05 GMT+02:00 Machiel mach...@integrat.co.za mach...@integrat.co.za: Hi Guys I have installed bacula7.0.5 from source as well as bacula-web today without any issues. I am however trying to install bacula-gui-7.0.5 , but when I run the configure script I get the following: ./configure --with-bacula=/installs/bacula-7.0.5/ configuring for Bacula source 7.0.5 01Jan13 29Feb12 4.8.4 (28 July 2014) checking for true... /bin/true checking for false... /bin/false checking for gawk... gawk checking for gawk... /usr/bin/gawk checking for a BSD-compatible install... /usr/bin/install -c checking for mv... /bin/mv checking for rm... /bin/rm checking for cp... /bin/cp checking for echo... /bin/echo checking for ar... /usr/bin/ar checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking for Operating System Distribution... done configure: creating ./config.status config.status: creating autoconf/Make.common sed: file ./confstatvUTcN2/subs-1.sed line 35: unterminated `s' command config.status: creating Makefile sed: file ./confstatvUTcN2/subs-1.sed line 35: unterminated `s' command config.status: creating bimagemgr/Makefile config.status: error: cannot find input file: bimagemgr/Makefile.in I tried the same on another machine running 7.0.4 and got the exact same thing. (thought it might be a big with the latest version). I am currently running ubuntu 14.04 LTS server and need to get this up and running soon. Any help would be greatly appreciated. Hello, In bacula-gui archive there is one webGUI tool named Baculum. Script ../configure is not used for Baculum installation. For installing Baculum please look in next file: bacula-gui-7.0.5/baculum/INSTALL There is description about how to install Baculum. Best regards. Marcin Haba -- -- Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich. Jezus Chrystus -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Error when trying to configure bacula-gui-7.0.5
Hi, Thank you for the reply. ok, two questions here. - I have only copied the baculum directory from the bacula-gui-7.0.5 source directory and to the /var/www/html/baculum directory. -- first question, am I supposed to copy the full bacula-gui directory or was this correct. - my current virtualhost config is as follows: VirtualHost *:80 DocumentRoot /var/www/html/baculum/ serveralias # hashed out info ServerName # hashed out info Directory /var/www/html/baculum RewriteEngine On AllowOverride All AuthType Basic AuthName MyPrivateFile AuthUserFile /var/www/html/baculum/passwords Require valid-user /Directory /VirtualHost This is the only site that will be running on the server so I have placed this virtualhost in the default file. If I need to put a rewrite rule in, then my second question is what would it look like as I dont know how to write rewrite_rules at all. Regards On 05/08/2014 14:20, Marcin Haba wrote: Hello, It looks that your DocumentRoot option from VirtualHost does not indicate to directory with Baculum files. The most probably is that you used subdirectory for storing Baculum files. For example, if you are storing Baculum files in location: /some/dir/bacula-gui/baculum then according Apache configuration file part can look like below: VirtualHost *:80 DocumentRoot /some/dir/bacula-gui/baculum ServerName somehost.example.com http://somehost.example.com Directory /some/dir/bacula-gui/baculum AllowOverride All AuthType Basic AuthName MyPrivateFile AuthUserFile /etc/apache2/some_password_file Require valid-user /Directory /VirtualHost In other words, Baculum files need to be located on top location of your web server document root. From this reason access to Baculum is by: http://yourhost/ but NOT by: http://yourhost/baculum/ I believe that this problem is easy to skip by modifing mod_rewrite rules. Best regards. Marcin 2014-08-05 10:30 GMT+02:00 Machiel mach...@integrat.co.za mailto:mach...@integrat.co.za: Hi Guys Thank you for the response on this matter. I have now followed the instructions for installing baculum as per the INSTALL file, however now I a getting the following when I go to the web interface: The requested URL /wizard/ was not found on this server. I have done some web searches and everyone is noting that the authorisation needs to be configured and that mod_rewrite needs to be enabled. Both these have been done, however still getting that message. Any ideas on what might be wrong or how to fix this? I am by no means an apache or php expert. Regards On 31/07/2014 17:52, Marcin Haba wrote: 2014-07-31 15:05 GMT+02:00 Machielmach...@integrat.co.za mailto:mach...@integrat.co.za: Hi Guys I have installed bacula7.0.5 from source as well as bacula-web today without any issues. I am however trying to install bacula-gui-7.0.5 , but when I run the configure script I get the following: ./configure --with-bacula=/installs/bacula-7.0.5/ configuring for Bacula source 7.0.5 01Jan13 29Feb12 4.8.4 (28 July 2014) checking for true... /bin/true checking for false... /bin/false checking for gawk... gawk checking for gawk... /usr/bin/gawk checking for a BSD-compatible install... /usr/bin/install -c checking for mv... /bin/mv checking for rm... /bin/rm checking for cp... /bin/cp checking for echo... /bin/echo checking for ar... /usr/bin/ar checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking for Operating System Distribution... done configure: creating ./config.status config.status: creating autoconf/Make.common sed: file ./confstatvUTcN2/subs-1.sed line 35: unterminated `s' command config.status: creating Makefile sed: file ./confstatvUTcN2/subs-1.sed line 35: unterminated `s' command config.status: creating bimagemgr/Makefile config.status: error: cannot find input file: bimagemgr/Makefile.in I tried the same on another machine running 7.0.4 and got the exact same thing. (thought it might be a big with the latest version). I am currently running ubuntu 14.04 LTS server and need to get this up and running soon. Any help would be greatly appreciated. Hello, In bacula-gui archive there is one webGUI tool named Baculum. Script ../configure is not used for Baculum installation. For installing Baculum please look in next file: bacula-gui-7.0.5/baculum/INSTALL
Re: [Bacula-users] Recycling issue
I’m also running into this issue.
In my case, I’ve setup a disk-based auto-changer (vchanger) with two pools:
Full and Weekly. Each pool has multiple drives with 8 volumes on each drive.
The script that mounts the disk also issues an “update slots” command in
bconsole, and I can verify that bacula sees the correct volumes as “InChanger”.
When I swap out disks (weekly), Bacula continues to look for volumes that were
append-able, but on the previous disk (and not currently marked as “InChanger”)
and refuses to write to the append-able volumes on the new disk that are marked
as “InChanger”. It marks all the previous volumes it cannot access as “Error”,
and only then will it begin writing to the currently loaded volumes.
This line from the “Recycling Algorithm” led me to believe that it would use a
volume that was InChanger first: If the request is for an Autochanger device,
look only for Volumes in the Autochanger (i.e. with InChanger set and that have
the correct Storage device).”
But it seems I’m missing something. I’ve included relevant bits from my
bacula-dir.conf file if anyone can see what I’m doing wrong. I’m getting this
on two systems, one running Bacula 5.2.12 and one running 7.0.4.
My other option is to split these all into separate pools, but that’s going to
make scheduling much harder.
Thanks in advance!
-Joe Rhodes
# Definition of file Virtual Autochanger device
Storage {
Name = removeable-drives
Address = 10.11.1.2
SDPort = 9103
Password = “password
Device = ExternalDrives
Media Type = File
Maximum Concurrent Jobs = 1
}
# Pools
Pool {
Name = FullSet
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Use Duration = 1 week
#We're on an 2 month rotation
VolumeRetention = 58 days
Maximum Volumes = 32 # 4 drives with 8 volumes on each
Storage = removeable-drives
}
Pool {
Name = Weekly
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Use Duration = 1 week
VolumeRetention = 3 weeks
Maximum Volumes = 40 # 5 drives with 8 volumes on each
Storage = removeable-drives
}
And my bacula-sd.conf:
# Define a Virtual autochanger
Autochanger {
Name = ExternalDrives
Device = ExternalDrives-Drive1
Changer Command = /usr/local/bin/vchanger -u root %c %o %S %a %d
Changer Device = /etc/bacula/vchanger1.conf
}
Device {
Name = ExternalDrives-Drive1
DriveIndex = 0
Autochanger = yes
DeviceType = File
Media Type = File
Archive Device = /var/lib/bacula/removeable-drives/0/drive0
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5;
Maximum Volume Size = 476250 M; # with 8 volumes per drive, this should
leave about
# 4,800 MB free for other things, or about
0.125%
}
From: Gael Guilmin gael.guil...@pdgm.com
Subject: [Bacula-users] Recycling issue
To: bacula-users@lists.sourceforge.net
bacula-users@lists.sourceforge.net
Message-ID:
05520d1d14f2408f8d048f1e1dfe6...@blupr07mb836.namprd07.prod.outlook.com
Content-Type: text/plain; charset=us-ascii
Hello,
I'm facing an issue with Bacula (5.2.12) on a Ubuntu server (12.04.4 LTS): I
don't know why, but the server, after finishing writing on a tape, tries to
use a tape that is not in the autochanger, despite the fact that there are
plenty of online and usable tape in the Scratch pool...
Any idea?
Best regards,
Gael
--
System Administrator / IT
[PDM_Logo_RGB - registered]
78, Avenue du XXeme Corps
54000 Nancy
France
Phone(Paris): +33 (0)1 49 49 05 25
Phone(Nancy): +33 (0)3 83 67 66 39
Cell: +33 (0)6 10 18 48 43
Web: www.pdgm.comhttp://www.pdgm.com/
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup on Tape
Hi John, We have MySql Database, is there document for tune database? Or Can you give us recommended setting for mysql? Thank you, Ankush -Original Message- From: John Drescher [mailto:dresche...@gmail.com] Sent: 04 August 2014 23:37 To: More, Ankush; Bacula Users Subject: Re: [Bacula-users] Backup on Tape 1) Can we tune bacula config files for maximize backup speed? Yes. I recommend that you tune your database (mysql and postgresql come with very poor default settings) and also increase the default block size. 64K is way too small for a modern LTO tape drive. 2) Can bacula use both tape drive simultaneous backup for same pool? Yes. It is possible to load more than 1 tape from the same pool to more than 1 tape drive and when these are loaded you can run different jobs on each tape drive. I do not remember the bacula directive to allow that however. Please google. This has been discussed dozens of times on this list. John This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup on Tape
On Tue, Aug 5, 2014 at 10:29 AM, More, Ankush ankush.m...@capgemini.com wrote: Hi John, We have MySql Database, is there document for tune database? Or Can you give us recommended setting for mysql? You may need to change memory settings for flushing, buffer sizes ... Some distributions ship with mysql optimized for a machine with 64MB of ram or something crazy like that. Yes I mean MB. Google search for mysql optimizing guides. The settings you need to change are not bacula specific. John -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup on Tape
On 04/08/14 18:54, Nick Allevato wrote: LTO 2…slow? Slow, but not as slow as mechanical disk drives. What _seriously_ slows down backups is disk seeking (both on the client array and on the spool disks), and tape drive will slow to a crawl if not fed at near-full speed. If you're backing up to LTO of any type, the only way to get reasonable performance is to backup to a high performance SSD spool (or SSD stripe) and then despool from that to the tape. A decently fast SSD spool area can handle simultaneous spool/depool operations - but do NOT try this with low end consumer SSDs, the write latencies will kill performance (most consumer SSD random write performance is so-so at best.). -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Disk based backup using vchanger, volumes being marked as Error
On 08/05/2014 02:10 PM, Josh Fisher wrote: On 8/5/2014 1:36 AM, Kern Sibbald wrote: Hello Josh, Please see below ... On 08/04/2014 06:43 PM, Josh Fisher wrote: On 8/1/2014 12:27 PM, Joseph Dickson wrote: Greetings :-) I've run into this problem with Bacula in a previous installation, and I can't seem to recall if there was ever a resolution.. I'm using Bacula for disk based backups only, and I am using vchanger to manage my virtual library. I've configured a vchanger library with 100 slots and 8 drives, and have set a Maximum Volume Bytes of 100G on the pool definition that I am using, to limit each slot in the library to 100G. I have also set a Maximum Concurrent Jobs = 2 setting on each of the virtual tape drive devices in my storage director config, so that only two jobs can write to a device at a time to minimize interleaving. Everything works perfectly as long as I only kick a few jobs off at a time.. however, when my main backup windows run and 30 or 40 backup jobs kick off, I often end up with jobs that output the following sequence in the logs: Have you set PreferMountedVolumes=no in the Job resource in bacula-dir.conf? If 3 jobs start and want to write to volumes in the same pool, then all three can be assigned the same volume. In fact, if PreferMountedVolumes=yes, (the default), then all three WILL be assigned the same volume unless the pool restricts the max number of jobs that the volume may contain. However, your device (drive) restricts the max concurrent jobs to 2. Therefore one of those three jobs will not be able to select the drive where the volume is mounted and will be forced to select another unused drive. That third job will nevertheless select the same volume as the other two and attempt to move the volume from the drive it is in into the drive that it has been assigned to. The configuration has a built-in race condition. I have recently done quite a bit of work to try to avoid race conditions such as the one you describe above. Does this still happen on version 7.0.x? I ask because there is now code that *should* detect this and explicitly makes the third job (as you describe above) wait. Now it is possible that there is some code path in the SD where the new code does not apply, so I cannot exclude problems, but if any exist in 7.0.x I would like to know so I can work on it some more. With the new code, the Volume will be moved around, but at least it should be done correctly without some deadlock or failure. I haven't had a chance to update to 7.0.x yet, so I can't say. My thought is that the volume itself should have a "Maximum Concurrent Jobs" setting, in addition to the SD Device. Better still, it could be automated by forcing the volume's max concurrency to that of the SD device at mount time. That should eliminate the need for "Prefer Mounted Volumes" altogether, since once the "Maximum Concurrent Jobs" have selected the volume, subsequent jobs would reject it as unavailable and so see the drive it is mounted in as unavailable at drive selection time. Once a drive is selected, that volume would be viewed as unavailable and rejected during volume selection, at least until one of the jobs using the volume ends. So by setting "Max Concurrent Jobs" to 1, one could guarantee a volume would never be selected by more than one job at a time. Yes, I would like to do something like what you say, but unfortunately I concerned that it opens up other possibilities for race conditions due to the fact that there are 3 components dealing with the data (SD, DIR, and the catalog). I have some ideas, and I hope to implement them in the next major Bacula version. Best regards, Kern ... -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future.
Re: [Bacula-users] Backup on Tape
On 08/05/2014 10:32 AM, Alan Brown wrote: If you're backing up to LTO of any type, the only way to get reasonable performance is to backup to a high performance SSD spool (or SSD stripe) and then despool from that to the tape. A decently fast SSD spool area can handle simultaneous spool/depool operations - but do NOT try this with low end consumer SSDs, the write latencies will kill performance (most consumer SSD random write performance is so-so at best.). I'd also consider write wear-out and your retention periods 'cause when you fry your system drive/bacula database ssd, you may have to bscan N years worth of tapes to make your backups accessible again. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu signature.asc Description: OpenPGP digital signature -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] lzo link error on static client
I am encountering a link error with the Bacula 7.0.4 static client and I don’t know how to fix this. long version: https://redports.org//~dvl/20140805183033-13761-230584/bacula-client-static-7.0.4_1.log Short version: Linking bacula-fd ... /usr/bin/c++ -L/usr/local/lib -liconv -L../lib -L../findlib -o bacula-fd filed.o authenticate.o acl.o backup.o estimate.o fd_plugins.o accurate.o filed_conf.o heartbeat.o job.o restore.o status.o verify.o verify_vol.o xattr.o -lz -lbacfind -lbaccfg -lbac -lm -lpthread -lintl /usr/local/lib/libintl.so -Wl,-rpath -Wl,/usr/local/lib -lssl -lcrypto -llzo2 /usr/bin/c++ -L/usr/local/lib -liconv -static -L../lib -L../findlib -o static-bacula-fd filed.o authenticate.o acl.o backup.o estimate.o fd_plugins.o accurate.o filed_conf.o heartbeat.o job.o restore.o status.o verify.o verify_vol.o xattr.o-lz -lbacfind -lbaccfg -lbac -lm -lpthread -lintl /usr/local/lib/libintl.a -Wl,-rpath -Wl,/usr/local/lib -lssl -lcrypto backup.o: In function `blast_data_to_storage_daemon(JCR*, char*)': backup.c:(.text+0xd64): undefined reference to `__lzo_init_v2' backup.o: In function `save_file(JCR*, FF_PKT*, bool)': backup.c:(.text+0x2b4e): undefined reference to `lzo1x_1_compress' restore.o: In function `decompress_data(JCR*, int, char**, unsigned int*)': restore.c:(.text+0x457): undefined reference to `lzo1x_decompress_safe' restore.o: In function `do_restore(JCR*)': restore.c:(.text+0x1eab): undefined reference to `__lzo_init_v2' /usr/local/lib/libintl.a(dcigettext.o): In function `_nl_find_msg': dcigettext.c:(.text+0x94b): undefined reference to `libiconv_open' dcigettext.c:(.text+0x9fa): undefined reference to `libiconv' dcigettext.c:(.text+0xbaf): undefined reference to `libiconv_open' /usr/local/lib/libintl.a(relocatable.o): In function `libintl_set_relocation_prefix': relocatable.c:(.text+0x14b): undefined reference to `libiconv_set_relocation_prefix' *** [static-bacula-fd] Error code 1 1 error — Dan Langille signature.asc Description: Message signed with OpenPGP using GPGMail -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] lzo link error on static client
On Aug 5, 2014, at 4:20 PM, Dan Langille d...@langille.org wrote: I am encountering a link error with the Bacula 7.0.4 static client and I don’t know how to fix this. long version: https://redports.org//~dvl/20140805183033-13761-230584/bacula-client-static-7.0.4_1.log This problem was caused by not using a clean environment. I had many libraries already installed. Working in a clean space gave success: https://redports.org/~dvl/20140805224410-97410-230652/bacula-client-static-7.0.4_1.log Thank you. — Dan Langille signature.asc Description: Message signed with OpenPGP using GPGMail -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Recycling issue
Hello,
Probably 95% of all such problems are due to configurations where
the MediaType is not properly set.
- If you are using any form of autochanger, each autochanger must
have a *unique* MediaType.
- Each Device configuration in the SD must have a unique MediaType
for each different ArchiveDevice definition. Said the other way
around, each ArchiveDevice definition must have its own unique
MediaType.
The .conf files you list are incomplete, so it is not possible to
analyze them, but when I see: MediaType = File, that raises a red
flag that indicates that there may be other Storage devices with
the same MediaType, and if that is the case, sooner or later, you
will find that Bacula is very confused.
In the next Bacula version, I will probably modify the SD and
possibly the Dir to require unique MediaTypes, then this problem
will be resolved.
I don't usually supply "support" because I don't have the time,
but this particular problem interests me from the standpoint that
I would like to prevent it, but with no console/job output showing
the failures and incomplete DIR and SD confs I cannot do any more.
Best regards,
Kern
On 08/05/2014 03:08 PM, Joe Rhodes wrote:
I’m also running into this issue.
In my case, I’ve setup a disk-based auto-changer (vchanger)
with two pools: Full and Weekly. Each pool has multiple drives
with 8 volumes on each drive. The script that mounts the disk
also issues an “update slots” command in bconsole, and I can
verify that bacula sees the correct volumes as “InChanger”.
When I swap out disks (weekly), Bacula continues to look for
volumes that were append-able, but on the previous disk (and not
currently marked as “InChanger”) and refuses to write to the
append-able volumes on the new disk that are marked as
“InChanger”. It marks all the previous volumes it cannot access
as “Error”, and only then will it begin writing to the currently
loaded volumes.
This line from the “Recycling Algorithm” led me to believe
that it would use a volume that was InChanger first: "If the
request is for an Autochanger device, look only for Volumes in
the Autochanger (i.e. with InChanger set and that have the
correct Storage device).”
But it seems I’m missing something. I’ve included relevant
bits from my bacula-dir.conf file if anyone can see what I’m
doing wrong. I’m getting this on two systems, one running
Bacula 5.2.12 and one running 7.0.4.
My other option is to split these all into separate pools,
but that’s going to make scheduling much harder.
Thanks in advance!
-Joe Rhodes
# Definition of file Virtual Autochanger device
Storage {
Name = removeable-drives
Address = 10.11.1.2
SDPort = 9103
Password = “password"
Device = ExternalDrives
Media Type = File
Maximum Concurrent Jobs = 1
}
# Pools
Pool {
Name = FullSet
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Use Duration = 1 week
#We're on an 2 month rotation
VolumeRetention = 58 days
Maximum Volumes = 32 # 4 drives with 8 volumes on each
Storage = removeable-drives
}
Pool {
Name = Weekly
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Use Duration = 1 week
VolumeRetention = 3 weeks
Maximum Volumes = 40 # 5 drives with 8 volumes on each
Storage = removeable-drives
}
And my bacula-sd.conf:
# Define a Virtual autochanger
Autochanger {
Name = ExternalDrives
Device = ExternalDrives-Drive1
Changer Command = "/usr/local/bin/vchanger -u root %c %o %S
%a %d"
Changer Device = "/etc/bacula/vchanger1.conf"
}
Device {
Name = ExternalDrives-Drive1
DriveIndex = 0
Autochanger = yes
DeviceType = File
Media Type = File
Archive Device = /var/lib/bacula/removeable-drives/0/drive0
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5;
Maximum Volume Size = 476250 M; # with 8 volumes per drive,
this should leave about
Re: [Bacula-users] Disk based backup using vchanger, volumes being marked as Error
On 08/04/2014 06:43 PM, Josh Fisher wrote: ... Have you set PreferMountedVolumes=no in the Job resource in bacula-dir.conf? If 3 jobs start and want to write to volumes in the same pool, then all three can be assigned the same volume. In fact, if PreferMountedVolumes=yes, (the default), then all three WILL be assigned the same volume unless the pool restricts the max number of jobs that the volume may contain. However, your device (drive) restricts the max concurrent jobs to 2. Therefore one of those three jobs will not be able to select the drive where the volume is mounted and will be forced to select another unused drive. That third job will nevertheless select the same volume as the other two and attempt to move the volume from the drive it is in into the drive that it has been assigned to. The configuration has a built-in race condition. This is the first time that I have heard this explained so clearly. I am going to try to duplicate this problem now that you have so clearly explained it. By the way, I am not really sure I would classify this as a race condition, because theoretically the SD is not blocked, the third job just waits until the Volume is free (at least that is what I programmed). However, this is clearly very inefficient. I would like to fix this, but one must keep in mind one important difficulty with Bacula. The SD knows what is going on with Volumes, but the Dir does not, and it is the Dir that proposes Volumes to the SD. Currently there is no good atomic way to pass the information in the SD to the Dir so that it can make better decisions. So, with the (current) restraint that the solution must involve changing only the SD algorithm, how could one prevent this from happening? I have some ideas, but wonder what you think. Setting PreferMountedVolumes=no causes the three jobs to select a drive that is NOT already mounted with a volume from the pool. This allows jobs writing to the same pool to select different volumes from the pool, rather than all selecting the same next available volume. This has its own caveats. It doesn't necessarily prevent two jobs from selecting the same volume in some cases, meaning that they will want to swap the volume back and forth between drives, which is another type of race condition. I have used this method successfully for a pool containing full backups only by setting PreferMountedVolumes=no in the job resource and setting MaximumVolumeJobs=1 in the pool resource. Since Bacula selects the volume for a job in an atomic manner, this forces an exclusive set of volumes for each job, thus preventing the race condition. This means that concurrency is limited only by the number of drives, but at the "expense" of creating a greater number of smaller volume files. I quote "expense" because on a disk vchanger it isn't usually a big issue to have more volume files. Doing this with a tape autochanger would use a lot more tapes and be truly more expensive. Of course unlimited concurrency is theoretical, since the hardware limits the USEFUL concurrency. I really do not like the PreferMountedVolumes = No option (I have probably said this many times), but I find your use of it very well explained and very interesting. Best regards, Kern ... -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
