Re: [Bacula-users] Recycling issue
Hello Kern,
Thanks for your answer.
But I don’t think what’s you’re pointing is related to my case as I‘ve only one
autochanger.
To give more detail on my configuration:
· I make a daily incremental backup on different servers (Pool Default)
· Every weekend I make a full backup of the servers (Pool_Full_Week)
· I make a monthly full backup of servers (Pool_Full_Month)
Here are my pools definitions in bacula-dir.conf:
# Default pool definition
Pool {
Name = Default
Pool Type = Backup
Recycle = yes # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 30 days # one month
Cleaning Prefix = CLN
RecyclePool = Scratch
}
# Full pool Month definition
Pool {
Name = Pool_Full_Month
Pool Type = Backup
Recycle = yes # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 7 years # one year
Cleaning Prefix = CLN
RecyclePool = Scratch
}
# Full pool Week definition
Pool {
Name = Pool_Full_Week
Pool Type = Backup
Recycle = yes # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 1 years # one year
Cleaning Prefix = CLN
RecyclePool = Scratch
}
# Scratch pool definition
Pool {
Name = Scratch
Pool Type = Backup
Cleaning Prefix = CLN
}
Here is my Storage definition in bacula-dir.conf:
Storage {
Name = Autochanger
Address = 192.168.27.7 # name or IP address of the server that hosts the daemon
storage
SDPort = 9103
Password =
Device = Autochanger # the same as defined in bacula-sd.conf
Media Type = LTO-5
}
Here is my bacula-sd.conf:
Autochanger {
Name = Autochanger
Device = Drive-1
Changer Command = /etc/bacula/scripts/mtx-changer %c %o %S %a %d
Changer Device = /dev/sg2
}
Device {
Name = Drive-1 #
Drive Index = 0
Media Type = LTO-5
Archive Device = /dev/nst0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = yes;
RemovableMedia = yes;
RandomAccess = no;
Autochanger = yes;
LabelMedia = no;
}
I don’t get why bacula tries to use tape that are not in the autochanger,
beside the fact that there are a lot of usable tapes in it that are in the
Scratch pool…
Thanks for your help,
Gael Guilmin
From: Kern Sibbald [mailto:k...@sibbald.com]
Sent: Wednesday, August 06, 2014 7:42 AM
To: Joe Rhodes; bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] Recycling issue
Hello,
Probably 95% of all such problems are due to configurations where the MediaType
is not properly set.
- If you are using any form of autochanger, each autochanger must have a
*unique* MediaType.
- Each Device configuration in the SD must have a unique MediaType for each
different ArchiveDevice definition. Said the other way around, each
ArchiveDevice definition must have its own unique MediaType.
The .conf files you list are incomplete, so it is not possible to analyze them,
but when I see: MediaType = File, that raises a red flag that indicates that
there may be other Storage devices with the same MediaType, and if that is the
case, sooner or later, you will find that Bacula is very confused.
In the next Bacula version, I will probably modify the SD and possibly the Dir
to require unique MediaTypes, then this problem will be resolved.
I don't usually supply support because I don't have the time, but this
particular problem interests me from the standpoint that I would like to
prevent it, but with no console/job output showing the failures and incomplete
DIR and SD confs I cannot do any more.
Best regards,
Kern
On 08/05/2014 03:08 PM, Joe Rhodes wrote:
I’m also running into this issue.
In my case, I’ve setup a disk-based auto-changer (vchanger) with two pools:
Full and Weekly. Each pool has multiple drives with 8 volumes on each drive.
The script that mounts the disk also issues an “update slots” command in
bconsole, and I can verify that bacula sees the correct volumes as “InChanger”.
When I swap out disks (weekly), Bacula continues to look for volumes that were
append-able, but on the previous disk (and not currently marked as “InChanger”)
and refuses to write to the append-able volumes on the new disk that are marked
as “InChanger”. It marks all the previous volumes it cannot access as “Error”,
and only then will it begin writing to the currently loaded volumes.
This line from the “Recycling Algorithm” led me to believe that it would use a
volume that was InChanger first: If the request is for an Autochanger device,
look only for Volumes in the Autochanger (i.e. with InChanger set and that have
the correct Storage device).”
But it seems I’m missing something. I’ve included relevant bits from my
bacula-dir.conf file if anyone can see what I’m doing wrong. I’m
Re: [Bacula-users] Recycling issue
On Wed, 06 Aug 2014 07:41:59 +0200, Kern Sibbald said: In the next Bacula version, I will probably modify the SD and possibly the Dir to require unique MediaTypes, then this problem will be resolved. I hope that forcing users to change the configured MediaType won't cause a problem with old media (because the label contains the media type and won't match anymore). __Martin -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Recycling issue
On Wed, 6 Aug 2014 06:52:06 +, Gael Guilmin said:
Here is my Storage definition in bacula-dir.conf:
Storage {
Name = Autochanger
Address = 192.168.27.7 # name or IP address of the server that hosts the
daemon storage
SDPort = 9103
Password =
Device = Autochanger # the same as defined in bacula-sd.conf
Media Type = LTO-5
}
You need to add
Autochanger = yes
to this bacula-dir definition, otherwise Bacula will not apply the InChanger
logic. I think this is missing from most of the examples.
__Martin
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Recycling issue
Kern:
Thanks for the pointers. I absolutely understand why can’t provider support,
and I appreciate any help you can give.
Unfortunately, I’m not sure the MediaType is my answer. I do have two
auto-changers configured. (One’s for local backups, one if for off-site
backups.) Each auto-changer has a unique Media Type.
I just tried Martin Simmons’ suggestion of adding “Autochanger = yes” to the
bacula-dir.conf storage stanza. I will know on Friday night if that was indeed
the answer, but it certainly looks promising. Assuming this is supposed to
work, and I’m getting the same problem on two different installations, missing
something like that certainly seems like a reasonable explanation.
I didn’t include the full config files in an effort to keep my original post
from getting too long. My apologies.I’ve pulled some of the configuration
out into separate files in an effort to keep my config files more manageable.
I’m happy to provider more if it would be helpful. My bacula-sd.conf is as
follows:
Storage {
Name = mail.roundhouse.local-sd
SDPort = 9103 # Director's port
WorkingDirectory = /var/bacula
Pid Directory = /var/run
Maximum Concurrent Jobs = 20
Client Connect Wait = 60 seconds
}
Director {
Name = mail.roundhouse.local-dir
Password = “password
}
Director {
Name = mail.roundhouse.local-mon
Password = “password
Monitor = yes
}
# Define a Virtual autochanger
#
Autochanger {
Name = ExternalDrives
Device = ExternalDrives-Drive1
Changer Command = /usr/local/bin/vchanger -u root %c %o %S %a %d
Changer Device = /etc/bacula/vchanger1.conf
}
Device {
Name = ExternalDrives-Drive1
DriveIndex = 0
Autochanger = yes
DeviceType = File
Media Type = File
Archive Device = /var/lib/bacula/removeable-drives/0/drive0
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5;
Maximum Volume Size = 476250 M; # with 8 volumes per drive, this should
leave about
# 4,800 MB free for other things, or about
0.125%
}
Autochanger {
Name = LocalStorage
Device = LocalDrives-Drive1, LocalDrives-Drive2
Changer Command = /usr/local/bin/vchanger -u root %c %o %S %a %d
Changer Device = /etc/bacula/vchanger2.conf
}
Device {
Name = LocalDrives-Drive1
Device Type = File
Media Type = Local
Archive Device = /var/lib/bacula/LocalStorage/0/drive0
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
Maximum Volume Size = 237500 MB # We limit the pool to 24 volumes in the
director.conf and autochanger
# config file. We have a 6 TB drive system.
DriveIndex = 0
Autochanger = yes
}
Device {
Name = LocalDrives-Drive2
Device Type = File
Media Type = Local
Archive Device = /var/lib/bacula/LocalStorage/1/drive1
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Maximum Volume Size = 237500 MB # We limit the pool to 24 volumes in the
director.conf and autochanger
# config file. We have a 6 TB drive system.
DriveIndex = 1
Autochanger = yes
}
Messages {
Name = Standard
director = mail.roundhouse.local-dir = all
}
My bacula-dir.conf file:
Director {# define myself
Name = mail.roundhouse.local-dir
DIRport = 9101# where we listen for UA connections
QueryFile = /etc/bacula/query.sql
WorkingDirectory = /var/bacula
PidDirectory = /var/run
Maximum Concurrent Jobs = 20
Password = “password # Console password
Messages = Daemon
}
#
#
# Job Defaults
#
#
@|sh -c 'for f in /etc/bacula/JobDef/*.defaults ; do echo @${f} ; done'
#
#
# Jobs
#
#
Job {
Name = Server
Client = server
Type = Backup
Level = Full
SpoolAttributes = yes
Schedule = WeeklyCycle
Cancel Lower Level Duplicates = yes
Cancel Queued Duplicates = yes
Rerun Failed Levels = yes
Messages = Standard
Priority = 10
Write Bootstrap = /var/bacula/%c.bsr
Pool = FullSet
File Set = Linux Server
Storage = removeable-drives
Run Before Job = /etc/bacula/mount-drive.sh
}
# Include all the jobs listed in the jobs folder
@|sh -c 'for f in /etc/bacula/jobs/*.job ; do echo @${f} ; done'
# Backup the catalog database (after the nightly save)
Job {
Name = BackupCatalog
Pool = OnSite_A
Client = server
Type = Admin
#Level = Full
FileSet=Catalog
Schedule = WeeklyCycleAfterBackup
# This creates an ASCII copy of the mysql
Re: [Bacula-users] Recycling issue
Hello Martin, Bacula never checks the Media Type vs what is in the label. The problem for users who have not understood the importance of making the Media Types unique will be to modify their catalog database. Before making this change, I will provide some reasonable solution. Best regards, Kern On 08/06/2014 12:44 PM, Martin Simmons wrote: On Wed, 06 Aug 2014 07:41:59 +0200, Kern Sibbald said: In the next Bacula version, I will probably modify the SD and possibly the Dir to require unique MediaTypes, then this problem will be resolved. I hope that forcing users to change the configured MediaType won't cause a problem with old media (because the label contains the media type and won't match anymore). __Martin -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Disk based backup using vchanger, volumes being marked as Error
Kern, This is exactly the problems that myself and others have been reporting with autochanger and tape volumes. Thank you Josh for the very descriptive details. One additional issue that these race conditions can also create, if the 1st 2 jobs fill the volume that the 3rd job is waiting for, the 3rd job will fail when it finds that it has mounted a read-only volume. Patti Clark Linux System Administrator RD Systems Support Oak Ridge National Laboratory From: Kern Sibbald k...@sibbald.commailto:k...@sibbald.com Date: Wednesday, August 6, 2014 at 1:52 AM To: Josh Fisher jfis...@pvct.commailto:jfis...@pvct.com, bacula-users@lists.sourceforge.netmailto:bacula-users@lists.sourceforge.net bacula-users@lists.sourceforge.netmailto:bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Disk based backup using vchanger, volumes being marked as Error On 08/04/2014 06:43 PM, Josh Fisher wrote: ... Have you set PreferMountedVolumes=no in the Job resource in bacula-dir.conf? If 3 jobs start and want to write to volumes in the same pool, then all three can be assigned the same volume. In fact, if PreferMountedVolumes=yes, (the default), then all three WILL be assigned the same volume unless the pool restricts the max number of jobs that the volume may contain. However, your device (drive) restricts the max concurrent jobs to 2. Therefore one of those three jobs will not be able to select the drive where the volume is mounted and will be forced to select another unused drive. That third job will nevertheless select the same volume as the other two and attempt to move the volume from the drive it is in into the drive that it has been assigned to. The configuration has a built-in race condition. This is the first time that I have heard this explained so clearly. I am going to try to duplicate this problem now that you have so clearly explained it. By the way, I am not really sure I would classify this as a race condition, because theoretically the SD is not blocked, the third job just waits until the Volume is free (at least that is what I programmed). However, this is clearly very inefficient. I would like to fix this, but one must keep in mind one important difficulty with Bacula. The SD knows what is going on with Volumes, but the Dir does not, and it is the Dir that proposes Volumes to the SD. Currently there is no good atomic way to pass the information in the SD to the Dir so that it can make better decisions. So, with the (current) restraint that the solution must involve changing only the SD algorithm, how could one prevent this from happening? I have some ideas, but wonder what you think. Setting PreferMountedVolumes=no causes the three jobs to select a drive that is NOT already mounted with a volume from the pool. This allows jobs writing to the same pool to select different volumes from the pool, rather than all selecting the same next available volume. This has its own caveats. It doesn't necessarily prevent two jobs from selecting the same volume in some cases, meaning that they will want to swap the volume back and forth between drives, which is another type of race condition. I have used this method successfully for a pool containing full backups only by setting PreferMountedVolumes=no in the job resource and setting MaximumVolumeJobs=1 in the pool resource. Since Bacula selects the volume for a job in an atomic manner, this forces an exclusive set of volumes for each job, thus preventing the race condition. This means that concurrency is limited only by the number of drives, but at the expense of creating a greater number of smaller volume files. I quote exp ense because on a disk vchanger it isn't usually a big issue to have more volume files. Doing this with a tape autochanger would use a lot more tapes and be truly more expensive. Of course unlimited concurrency is theoretical, since the hardware limits the USEFUL concurrency. I really do not like the PreferMountedVolumes = No option (I have probably said this many times), but I find your use of it very well explained and very interesting. Best regards, Kern ... -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Disk based backup using vchanger, volumes being marked as Error
Hello Patti, Are you sure this is happening on 7.0.x? I worked hard on this problem between 5.2.x and 7.0.x, and I thought I had fixed the problem -- unless there is some code path I missed. It is possible that this is happening only when there is no other drive that the 3rd job can use, but if there are other drives available, Job 3 should select a different volume. Best regards, Kern On 08/06/2014 03:16 PM, Clark, Patricia A. wrote: Kern, This is exactly the problems that myself and others have been reporting with autochanger and tape volumes. Thank you Josh for the very descriptive details. One additional issue that these race conditions can also create, if the 1st 2 jobs fill the volume that the 3rd job is waiting for, the 3rd job will fail when it finds that it has mounted a read-only volume. Patti Clark Linux System Administrator RD Systems Support Oak Ridge National Laboratory From: Kern Sibbald k...@sibbald.commailto:k...@sibbald.com Date: Wednesday, August 6, 2014 at 1:52 AM To: Josh Fisher jfis...@pvct.commailto:jfis...@pvct.com, bacula-users@lists.sourceforge.netmailto:bacula-users@lists.sourceforge.net bacula-users@lists.sourceforge.netmailto:bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Disk based backup using vchanger, volumes being marked as Error On 08/04/2014 06:43 PM, Josh Fisher wrote: ... Have you set PreferMountedVolumes=no in the Job resource in bacula-dir.conf? If 3 jobs start and want to write to volumes in the same pool, then all three can be assigned the same volume. In fact, if PreferMountedVolumes=yes, (the default), then all three WILL be assigned the same volume unless the pool restricts the max number of jobs that the volume may contain. However, your device (drive) restricts the max concurrent jobs to 2. Therefore one of those three jobs will not be able to select the drive where the volume is mounted and will be forced to select another unused drive. That third job will nevertheless select the same volume as the other two and attempt to move the volume from the drive it is in into the drive that it has been assigned to. The configuration has a built-in race condition. This is the first time that I have heard this explained so clearly. I am going to try to duplicate this problem now that you have so clearly explained it. By the way, I am not really sure I would classify this as a race condition, because theoretically the SD is not blocked, the third job just waits until the Volume is free (at least that is what I programmed). However, this is clearly very inefficient. I would like to fix this, but one must keep in mind one important difficulty with Bacula. The SD knows what is going on with Volumes, but the Dir does not, and it is the Dir that proposes Volumes to the SD. Currently there is no good atomic way to pass the information in the SD to the Dir so that it can make better decisions. So, with the (current) restraint that the solution must involve changing only the SD algorithm, how could one prevent this from happening? I have some ideas, but wonder what you think. Setting PreferMountedVolumes=no causes the three jobs to select a drive that is NOT already mounted with a volume from the pool. This allows jobs writing to the same pool to select different volumes from the pool, rather than all selecting the same next available volume. This has its own caveats. It doesn't necessarily prevent two jobs from selecting the same volume in some cases, meaning that they will want to swap the volume back and forth between drives, which is another type of race condition. I have used this method successfully for a pool containing full backups only by setting PreferMountedVolumes=no in the job resource and setting MaximumVolumeJobs=1 in the pool resource. Since Bacula selects the volume for a job in an atomic manner, this forces an exclusive set of volumes for each job, thus preventing the race condition. This means that concurrency is limited only by the number of drives, but at the expense of creating a greater number of smaller volume files. I quote e xpense because on a disk vchanger it isn't usually a big issue to have more volume files. Doing this with a tape autochanger would use a lot more tapes and be truly more expensive. Of course unlimited concurrency is theoretical, since the hardware limits the USEFUL concurrency. I really do not like the PreferMountedVolumes = No option (I have probably said this many times), but I find your use of it very well explained and very interesting. Best regards, Kern ... -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future.
[Bacula-users] List files associated with destroyed tape
Greetings, Our ancient autoloader decided to render a particular tape inoperable. Before my time jobs were run that spanned multiple volumes, the destroyed tape was in one such job. I cannot figure out how to find the files associated with the specific media. Here's a query that I thought would work but returns the same for any media id provided: SELECT t3.Path, t1.Name FROM Filename AS t1 JOIN File as t2 ON t1.FilenameId = t2.FilenameId JOIN Path as t3 ON t3.PathId = t2.PathId JOIN JobMedia as t4 ON t2.JobId = t4.JobId WHERE t4.MediaId = 475 AND t2.JobId = 8876 LIMIT 10; Any help would be much appreciated. -- John Wallach Systems Administrator PGP Fingerprint: C089 65CA 1F30 8A02 86C5 6BC3 59E1 ACE4 F8B4 28DB Democracy Now! www.democracynow.org -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] List files associated with destroyed tape
On Aug 6, 2014, at 12:11 PM, John Wallach j...@democracynow.org wrote: Greetings, Our ancient autoloader decided to render a particular tape inoperable. Before my time jobs were run that spanned multiple volumes, the destroyed tape was in one such job. I cannot figure out how to find the files associated with the specific media. Here's a query that I thought would work but returns the same for any media id provided: SELECT t3.Path, t1.Name FROM Filename AS t1 JOIN File as t2 ON t1.FilenameId = t2.FilenameId JOIN Path as t3 ON t3.PathId = t2.PathId JOIN JobMedia as t4 ON t2.JobId = t4.JobId WHERE t4.MediaId = 475 AND t2.JobId = 8876 LIMIT 10; Any help would be much appreciated. Does this help? http://dan.langille.org/2009/08/28/what-jobs-are-on-these-tapes/ — Dan Langille signature.asc Description: Message signed with OpenPGP using GPGMail -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Disk based backup using vchanger, volumes being marked as Error
On 8/6/2014 1:52 AM, Kern Sibbald wrote: On 08/04/2014 06:43 PM, Josh Fisher wrote: ... Have you set PreferMountedVolumes=no in the Job resource in bacula-dir.conf? If 3 jobs start and want to write to volumes in the same pool, then all three can be assigned the same volume. In fact, if PreferMountedVolumes=yes, (the default), then all three WILL be assigned the same volume unless the pool restricts the max number of jobs that the volume may contain. However, your device (drive) restricts the max concurrent jobs to 2. Therefore one of those three jobs will not be able to select the drive where the volume is mounted and will be forced to select another unused drive. That third job will nevertheless select the same volume as the other two and attempt to move the volume from the drive it is in into the drive that it has been assigned to. The configuration has a built-in race condition. This is the first time that I have heard this explained so clearly. I am going to try to duplicate this problem now that you have so clearly explained it. By the way, I am not really sure I would classify this as a race condition, because theoretically the SD is not blocked, the third job just waits until the Volume is free (at least that is what I programmed). However, this is clearly very inefficient. I agree. It is not a race condition in the code at all. Nothing gets stuck. It is really a misconfiguration, though the config file is syntactically correct. I'm not sure what to call that. I suppose I should have said the configuration has a built-in "resource contention problem", rather than race condition. Sorry for the confusion. I would like to fix this, but one must keep in mind one important difficulty with Bacula. The SD knows what is going on with Volumes, but the Dir does not, and it is the Dir that proposes Volumes to the SD. Currently there is no good atomic way to pass the information in the SD to the Dir so that it can make better decisions. So, with the (current) restraint that the solution must involve changing only the SD algorithm, how could one prevent this from happening? I have some ideas, but wonder what you think. I think that it in fact MUST be changed only in the SD. The issue is that the volume selection for a job needs to be atomic. Whether the volume info is acquired from Dir, and array in SD, or anywhere else, SD must access it in a critical section in order to serialize volume selection. I believe that ANYTHING that changes the status of a volume or device should be handled in SD as an atomic operation. Consider a single mutex that must be held in order to make any changes to either a volume or a device. The status of devices and volumes is transmitted back to Dir as part of the mutex release. Dir then always has accurate info, because only one job at a time can change anything. (I also consider Dir commands to the SD to be "jobs" in this context). I guess I am of the belief that the current per-device locking is too fine grained. Due to volume selection, one device can affect another, even if indirectly as in the swapping required when the same volume is needed on two devices. A global lock simplifies concurrency and imho makes the whole system more robust. The biggest con is that multiple devices cannot mount/umount volumes at the same time. As far as I know, most tape robots cannot load/unload multiple drives simultaneously anyway, and for disk the mount/umount is only a few ms at most, so I don't view that as a problem. I think concurrent programming is just hard, period. :) Therefore I prefer simplifying the serialization over squeezing out the utmost performance. And I think a global acquisition lock in SD is the way to do that. Setting PreferMountedVolumes=no causes the three jobs to select a drive that is NOT already mounted with a volume from the pool. This allows jobs writing to the same pool to select different volumes from the pool, rather than all selecting the same next available volume. This has its own caveats. It doesn't necessarily prevent two jobs from selecting the same volume in some cases, meaning that they will want to swap the volume back and forth between drives, which is another type of race condition. I have used this method successfully for a pool containing full backups only by setting PreferMountedVolumes=no in the job resource and setting MaximumVolumeJobs=1 in the pool resource.
Re: [Bacula-users] List files associated with destroyed tape
On 8/6/14 12:33 PM, Dan Langille wrote: Does this help? http://dan.langille.org/2009/08/28/what-jobs-are-on-these-tapes/ Thanks for the suggestion but I already know the media that is lost, I guess I could purge the volume but I wonder what what that would do to the job (since it has files on other volumes.) -- John Wallach Systems Administrator PGP Fingerprint: C089 65CA 1F30 8A02 86C5 6BC3 59E1 ACE4 F8B4 28DB Democracy Now! www.democracynow.org -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] List files associated with destroyed tape
On Aug 6, 2014, at 12:11 PM, John Wallach j...@democracynow.org wrote: Greetings, Our ancient autoloader decided to render a particular tape inoperable. Before my time jobs were run that spanned multiple volumes, the destroyed tape was in one such job. I cannot figure out how to find the files associated with the specific media. Here's a query that I thought would work but returns the same for any media id provided: SELECT t3.Path, t1.Name FROM Filename AS t1 JOIN File as t2 ON t1.FilenameId = t2.FilenameId JOIN Path as t3 ON t3.PathId = t2.PathId JOIN JobMedia as t4 ON t2.JobId = t4.JobId WHERE t4.MediaId = 475 AND t2.JobId = 8876 LIMIT 10; SELECT t3.Path, t1.Name FROM Filename AS FN JOIN File as F ON FN.FilenameId = F.FilenameId JOIN Path as P ON P.PathId = F.PathId JOIN JobMedia as JB ON F.JobId = JB.JobId WHERE JB.MediaId = 475 AND F.JobId = 8876 LIMIT 10; Restructured the query to improve readability. Not tested. Combining your query and my blog post: You already know the Job: 8876. Mine is 181425 You already know the Volume name. Let’s get the volume id (mediaid): bacula=# select MediaId, VolumeName from Media where MediaId in (select distinct(MediaId) from JobMedia where JobId = 181425); mediaid | volumename -+- 2929 | IncrAutoNoNextPool-2929 (1 row) So just select the entries from the file table: bacula=# select fileid, filenameid, pathid from file where jobid = 181425 limit 10; fileid | filenameid | pathid ---++- 527206021 |8285311 | 1473156 527206020 |291 | 1473278 527206019 |2964380 | 1473278 527206018 |2964379 | 1473278 527206017 |2964378 | 1473278 527206016 |2964376 | 1473278 527206015 |291 | 1473025 527206014 |2964380 | 1473025 527206013 |2964379 | 1473025 527206012 |2964376 | 1473025 (10 rows) bacula=# Now let’s get the names and paths: SELECT F.fileid, F.filenameid, F.pathid, P.path || '/' || FN.name FROM file F JOIN filename FN on F.filenameid = FN.filenameid JOIN path P ON F.pathid = P.pathid WHERE jobid = 181425 LIMIT 10; bacula-# LIMIT 10; fileid | filenameid | pathid | ?column? ---++-+--- 527206021 |8285311 | 1473156 | usr/home/dan//.mailfilter.log 527206020 |291 | 1473278 | usr/home/dan/Maildir// 527206019 |2964380 | 1473278 | usr/home/dan/Maildir//dovecot.index.log 527206018 |2964379 | 1473278 | usr/home/dan/Maildir//dovecot.index.cache 527206017 |2964378 | 1473278 | usr/home/dan/Maildir//dovecot.index 527206016 |2964376 | 1473278 | usr/home/dan/Maildir//dovecot-uidlist 527206015 |291 | 1473025 | usr/home/dan/Maildir/.Trash// 527206014 |2964380 | 1473025 | usr/home/dan/Maildir/.Trash//dovecot.index.log 527206013 |2964379 | 1473025 | usr/home/dan/Maildir/.Trash//dovecot.index.cache 527206012 |2964376 | 1473025 | usr/home/dan/Maildir/.Trash//dovecot-uidlist (10 rows) bacula=# Hope that helps — Dan Langille signature.asc Description: Message signed with OpenPGP using GPGMail -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] List files associated with destroyed tape
On Aug 6, 2014, at 12:37 PM, John Wallach j...@democracynow.org wrote: On 8/6/14 12:33 PM, Dan Langille wrote: Does this help? http://dan.langille.org/2009/08/28/what-jobs-are-on-these-tapes/ Thanks for the suggestion but I already know the media that is lost, I guess I could purge the volume but I wonder what what that would do to the job (since it has files on other volumes.) It’s hard to extrapolate sometimes. See other email. The above [unasked] question is not what you originally asked, but I understand the concern. I’m not sure of the answer though. It would be a great trialerror task. If you can, try it on non-critial data instead. I’m interested in the results. — Dan Langille signature.asc Description: Message signed with OpenPGP using GPGMail -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] List files associated with destroyed tape
It’s hard to extrapolate sometimes. See other email. The above [unasked] question is not what you originally asked, but I understand the concern. I’m not sure of the answer though. It would be a great trialerror task. If you can, try it on non-critial data instead. I’m interested in the results. Sorry for being unclear about what I am attempting. I might make backup of catalog, query for all files associated with job then purge the volume and run same query and diff for results. But I dislike messing with the catalog in general. -- John Wallach Systems Administrator PGP Fingerprint: C089 65CA 1F30 8A02 86C5 6BC3 59E1 ACE4 F8B4 28DB Democracy Now! www.democracynow.org -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] List files associated with destroyed tape
On Wed, 06 Aug 2014 12:37:57 -0400, John Wallach said: On 8/6/14 12:33 PM, Dan Langille wrote: Does this help? http://dan.langille.org/2009/08/28/what-jobs-are-on-these-tapes/ Thanks for the suggestion but I already know the media that is lost, I guess I could purge the volume but I wonder what what that would do to the job (since it has files on other volumes.) Purging the volume will delete all of its jobs from the catalog, so the records for the other volumes will be updated appropriately. __Martin -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] New user with Bacula
So, jumping into this a bit late, but... About six years ago, I had a NetApp that I needed to back up using Bacula. I spent about a week scripting a solution using the hidden shell on the device. Solution used the NetApp 'dump' command piped out to Bacula using rsh (tried ssh, but the performance was horrible). Restore was the reverse: pipe back to the NetApp, through 'load' along with the parameter indicating what to extract. It worked well for a roll your own solution, but I don't know whether the later versions of their Filer OS still support the interface. _KMP On Wednesday, July 30, 2014, dweimer dwei...@dweimer.net wrote: On 07/30/2014 9:42 am, Dmitri Maziuk wrote: On 7/30/2014 7:56 AM, Radosław Korzeniewski wrote: Could you explain what NetApp Client is/will be/could be, please. What do you mean by NetApp Client. I'd like to understand this thread, but I'm not an english native speaker so I miss the main point of this thread. It's not your English, it's NetApp. ;) -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Configure Options of a compiled version
Hello! How do get the Configure Options of a compiled bacula version? Best Regards Alex -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
