[Bacula-users] Trouble getting jobs to run simultaneously
This is a very common issue that must be looked at carefully, this will be a large post and I apologize for it :P Priority levels are very hard to manage/maintain since they can cause jobs to block! Meaning if a job has network (or other) issues and the Bacula server connection doesn't drop but the client does; you could be in a situation which I have faced numerous times over the last 3 years where over a weekend only the higher priority or equal to jobs will backup but others become stuck. I still use priorities (ex: catalog backup is 99, others are 1 to 10) but I've got other safe guards in place such as SD/FD timeouts to enforce job rescheduling (it requires you actually setup rescheduling parameters though and understand them, canceling a job may also reschedule it too - be warned) There's another gotcha that can be seen more with this sort of configuration (using priorities) where Bacula is trying to contact a client who behave well (port not open, firewalled, whatever) you may need to use a port checking program to check the remote host/port to see if it is open and if not error the job via runscript option (this is ran from the Bacula server, not client obviously), *this* problem I've had at least for over a year or so and was the only way to solve that specific issue. After doing all of the above priorities seem to work now... Heres the options you may want to look at for Job or JobDefs - this is straight from my config: Allow Mixed Priority = yes # Cancel, Error, Long running, Duplicate Job and Rerun control's go here Allow Duplicate Jobs = no Cancel Running Duplicates = yes # avoid weekend queue build ups... Cancel Queued Duplicates = yes Rerun Failed Levels = yes Reschedule On Error = yes Reschedule Interval = 15n #15n = 15 minute (most jobs reschedule after an hour, not this one) # Manually canceling a job may require canceling as many times as Reschedule times if set for. # Meaning it has to reschedule the job and you have to cancel it over and over until it is really considered cancelled... - Reschedule Times = 3 # 3 * Interval # if over 6h the job should be considered stuck, use this with caution! Max Run Time = 6h # the most important option to look at, what if we miss a full due to some error? # this needs to be set per client by looking at the schedule and deciding on best number to set: Max Full Interval=7 days +-- |This was sent by ccs...@hotmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Trouble getting jobs to run simultaneously
All,
Recently I've made changes to my bacula-dir and bacula-sd to with the intent of
running simultaneous jobs. After the changes my testing seems to indicate I
still don't have it right. I launched my first job 'Image_Backup' manually
with it's default priority, then launched the retail job modifying the priority
to 1, with the hope to force a simultaneous backup. The retail job sat waiting
for the image job to complete. I've attached the relevant sections from my
bacula-dir.conf. If anyone has any helpful thoughts I'd really appreciate it.
Troy
bacula-server-5.2.6
excerpt from bacula-dir.conf:
Pool{
Name = ImageF
Pool Type = Backup
Volume Use Duration = 7d
Volume Retention = 24 months
Catalog Files = yes
Label Format = ImageF-
Recycle = yes
AutoPrune = yes
}
Pool
{
Name = RetailF
Pool Type = Backup
Maximum Volume Jobs = 4
Volume Use Duration = 2d
Volume Retention = 6 months
Catalog Files = yes
Label Format = RetailF-
Recycle = yes
AutoPrune = yes
}
Job {
Name = Image_Backup
Enabled = YES
Type = Backup
Level= Full
Accurate= yes #[Yes|No]
Client = krusty-fd
Fileset = krusty-image
Max Wait Time = 72000
Schedule = WeeklyCycle
Storage = krustykrab-6
Messages = Standard
Pool = ImageF
Write Bootstrap = /data/working/image.bsr
Priority = 50
}
Job {
Name = Retail_Backup
Enabled = Yes
Type = Backup
Level= Full
Accurate= yes
Client = retail-fd
Fileset = retail
Max Wait Time = 72000
Schedule = WeeklyCycle
Storage = krustykrab-5
Messages = Standard
Pool = RetailF
Write Bootstrap = /data/working/retail.bsr
Priority = 60
}
FileSet{
Name = krusty-image
Include {
Options {
Compression = GZIP
signature = SHA1
Sparse = yes
Hardlinks = no
}
@/usr/local/etc/backuplist/krusty_image_backuplist
}}
FileSet{
Name = retail
Include {
Options {
Compression = GZIP
signature = SHA1
Sparse = yes
Hardlinks = no
}
@/usr/local/etc/backuplist/Retail_backuplist
}
Exclude {
File=/usr/home/sessions
}}
Storage {
Name = krustykrab-6
Address = krustykrab
SDPort = 9103
Password = removed
Device = krustykrabpool-6
Media Type = File-6
Maximum Concurrent Jobs = 20
}
Storage {
Name = krustykrab-5
Address = krustykrab
SDPort = 9103
Password = password
Device = krustykrabpool-5
Media Type = File-5
Maximum Concurrent Jobs = 20
}
excerpt from bacula-sd.conf
Device {
Name = krustykrabpool-5
Media Type = File-5
Archive Device = /data/Retail
LabelMedia = yes;
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = yes;
}
Device {
Name = krustykrabpool-6
Media Type = File-6
Archive Device = /data/Image
LabelMedia = yes;
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = yes;
}
_
Scanned by IBM Email Security Management Services
powered by MessageLabs.
_
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Trouble getting jobs to run simultaneously
Zitat von Troy Kocher tkoc...@mtadistributors.com: All, Recently I've made changes to my bacula-dir and bacula-sd to with the intent of running simultaneous jobs. After the changes my testing seems to indicate I still don't have it right. I launched my first job 'Image_Backup' manually with it's default priority, then launched the retail job modifying the priority to 1, with the hope to force a simultaneous backup. The retail job sat waiting for the image job to complete. I've attached the relevant sections from my bacula-dir.conf. If anyone has any helpful thoughts I'd really appreciate it. Troy Hello Only Jobs with the *same* priority run concurrently if all resources used support concurrent use. The value of priority is only used to decide which Job to start first if more than one is scheduled at the same time. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Trouble getting jobs to run simultaneously
You can also set Allow Mixed Priority = yes on all your jobs. From bacula docs: Allow Mixed Priority = yesno This directive is only implemented in version 2.5 and later. When set to yes (default no), this job may run even if lower priority jobs are already running. This means a high priority job will not have to wait for other jobs to finish before starting. The scheduler will only mix priorities when all running jobs have this set to true. Note that only higher priority jobs will start early. Suppose the director will allow two concurrent jobs, and that two jobs with priority 10 are running, with two more in the queue. If a job with priority 5 is added to the queue, it will be run as soon as one of the running jobs finishes. However, new priority 10 jobs will not be run until the priority 5 job has finished. -- Diego P. Rubert Coordenadoria de Informtica Procuradoria da Repblica no Estado de Mato Grosso do Sul Ministrio Pblico Federal Em 06-12-2012 18:13, lst_ho...@kwsoft.de escreveu: Zitat von Troy Kocher tkoc...@mtadistributors.com: All, Recently I've made changes to my bacula-dir and bacula-sd to with the intent of running simultaneous jobs. After the changes my testing seems to indicate I still don't have it right. I launched my first job 'Image_Backup' manually with it's default priority, then launched the retail job modifying the priority to 1, with the hope to force a simultaneous backup. The retail job sat waiting for the image job to complete. I've attached the relevant sections from my bacula-dir.conf. If anyone has any helpful thoughts I'd really appreciate it. Troy Hello Only Jobs with the *same* priority run concurrently if all resources used support concurrent use. The value of priority is only used to decide which Job to start first if more than one is scheduled at the same time. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
