Re: [Bacula-users] Running Copy Job Immediately After Disk Backups

[Phil Stracchino]

On 2020-04-06 14:52, Pierre Bernhardt wrote:
> Hi,
> 
> schedule your jobs with prio 10 for parallel running.
> schedule a job which starts the copy job with prio >= 11 so it runs after
> all prio 10 jobs has been finished which start the copy job.
> 
> Hope it will work as expected ;-)


No, this will not work.  The execution parameters for a Copy job —
including which jobs it is to copy — are evaluated not at the time the
job actually begins to run, but when it is queued to run, and running
jobs which have not yet completed will not be considered for copying.
The way the selection code currently works, you can schedule and queue a
Copy job only after all of the jobs you want it to copy have completed.



-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Running Copy Job Immediately After Disk Backups

[Pierre Bernhardt]

Hi,

schedule your jobs with prio 10 for parallel running.
schedule a job which starts the copy job with prio >= 11 so it runs after
all prio 10 jobs has been finished which start the copy job.

Hope it will work as expected ;-)

Cheers,



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Running Copy Job Immediately After Disk Backups

[Phil Stracchino]

On 2020-04-06 09:41, Richard Laysell wrote:
> 
> Hello,
> 
> I would like to run a copy job immediately after a set of disk backups
> has completed, but this seems to not work as I expect.
> 
> I am scheduling the jobs as follows
> 
> Client1 - Disk backup - Priority 10
> Client2 - Disk backup - Priority 10
> Client3 - Disk backup - Priority 10
> Client4 - Disk backup - Priority 10
> Client5 - Disk backup - Priority 10
> and so on for other clients
> All of these jobs will run in parallel (up to the concurrency limit)
> 
> Copy Job - Disk to Tape - Priority 20 - Pool Uncopied Jobs.
> This job should backup all of the just completed disk backups to a tape
> which goes offsite.  This gives both onsite and offsite copies.
> 
> When I try to implement this, the disk backups all complete
> successfully, but the copy job does not copy anything to tape.  After
> analysing this I think it is because the copy job looks for jobs to
> copy as soon as it starts.  This means that if it is started at the
> same time or while any of the disk jobs are running it will copy only
> the completed ones even thought it does not actually run until all of
> the disk backups have completed (because it has a lower priority)

I am familiar with this problem.  The root cause of the problem is that
the list of jobs for a copy job to copy is evaluated when it is
scheduled to run, not when it actually becomes runnable and begins
executing.  This means that by the time the job actually runs, it may be
running based upon stale data, as you have discovered.


> I could schedule the copy job to start an hour after I expect the
> latest disk backup to finish but this means that the backups do not
> complete as early as possible.

This is the only practical workaround that I have so far found.

> Is it possible to do one of the following:
> 1. Delay the selection of jobs for the copy job until it actually runs
>rather than do the selection when it initially starts

There has been some discussion of exactly this, which would actually
make copy jobs more consistent with the behavior of other job types as
well as being more behaviorally correct.

> 2. Have a job (priority 20) which starts the copy job.  This would then
>start the copy job only when all of the disk backups are complete.

If you come up with a clean and practical way to do this, I'd be
interested to see it.




-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Running Copy Job Immediately After Disk Backups

[Heitor Faria]

Hello Richard

Maybe you can deploy an After Job script that can start the copy of the 
original backup job after a few seconds:

run job=copy_job jobid=xx yes | bconsole

Regards,
--
MSc Heitor Faria
CEO Bacula LatAm
mobile1: + 1 909 655-8971
mobile2: + 55 61 98268-4220

América Latina
[ http://bacula.lat/]

 Original Message 
From: Richard Laysell 
Sent: Monday, April 6, 2020 10:22 AM
To: Bacula 
Subject: [Bacula-users] Running Copy Job Immediately After Disk Backups

>
>Hello,
>
>I would like to run a copy job immediately after a set of disk backups
>has completed, but this seems to not work as I expect.
>
>I am scheduling the jobs as follows
>
>Client1 - Disk backup - Priority 10
>Client2 - Disk backup - Priority 10
>Client3 - Disk backup - Priority 10
>Client4 - Disk backup - Priority 10
>Client5 - Disk backup - Priority 10
>and so on for other clients
>All of these jobs will run in parallel (up to the concurrency limit)
>
>Copy Job - Disk to Tape - Priority 20 - Pool Uncopied Jobs.
>This job should backup all of the just completed disk backups to a tape
>which goes offsite.  This gives both onsite and offsite copies.
>
>When I try to implement this, the disk backups all complete
>successfully, but the copy job does not copy anything to tape.  After
>analysing this I think it is because the copy job looks for jobs to
>copy as soon as it starts.  This means that if it is started at the
>same time or while any of the disk jobs are running it will copy only
>the completed ones even thought it does not actually run until all of
>the disk backups have completed (because it has a lower priority)
>
>I could schedule the copy job to start an hour after I expect the
>latest disk backup to finish but this means that the backups do not
>complete as early as possible.
>
>Is it possible to do one of the following:
>1. Delay the selection of jobs for the copy job until it actually runs
>   rather than do the selection when it initially starts
>2. Have a job (priority 20) which starts the copy job.  This would then
>   start the copy job only when all of the disk backups are complete.
>3. Something else that I have not thought of which fixes this issue.
>
>Regards,
>
>Richard
>
>
>___
>Bacula-users mailing list
>Bacula-users@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/bacula-users
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Running Copy Job Immediately After Disk Backups

[Richard Laysell]

Hello,

I would like to run a copy job immediately after a set of disk backups
has completed, but this seems to not work as I expect.

I am scheduling the jobs as follows

Client1 - Disk backup - Priority 10
Client2 - Disk backup - Priority 10
Client3 - Disk backup - Priority 10
Client4 - Disk backup - Priority 10
Client5 - Disk backup - Priority 10
and so on for other clients
All of these jobs will run in parallel (up to the concurrency limit)

Copy Job - Disk to Tape - Priority 20 - Pool Uncopied Jobs.
This job should backup all of the just completed disk backups to a tape
which goes offsite.  This gives both onsite and offsite copies.

When I try to implement this, the disk backups all complete
successfully, but the copy job does not copy anything to tape.  After
analysing this I think it is because the copy job looks for jobs to
copy as soon as it starts.  This means that if it is started at the
same time or while any of the disk jobs are running it will copy only
the completed ones even thought it does not actually run until all of
the disk backups have completed (because it has a lower priority)

I could schedule the copy job to start an hour after I expect the
latest disk backup to finish but this means that the backups do not
complete as early as possible.

Is it possible to do one of the following:
1. Delay the selection of jobs for the copy job until it actually runs
   rather than do the selection when it initially starts
2. Have a job (priority 20) which starts the copy job.  This would then
   start the copy job only when all of the disk backups are complete.
3. Something else that I have not thought of which fixes this issue.

Regards,

Richard


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Can't make brestore work. SOLVED.

[Erik P. Olsen]

On 2020-04-05 at 16:35:54 Erik P. Olsen wrote:

> On 2020-04-05 at 23:26:09 Gary R. Schmidt wrote:
> 
> > On 05/04/2020 22:52, Erik P. Olsen wrote:  
> > > Hello,
> > > 
> > > Where can I find description on how to use bat/brestore? I haven't so far 
> > > been able
> > > to restore using bat. I invariably end up getting error message from the 
> > > restore job
> > > saying: level = "" not valid. I don't find any help in the various 
> > > manuals and
> > > neither by googling.
> > > 
> > Does it work if you do it using bconsole?  
> 
> Yes it does.
> 

Problem was incorrect dir.conf. I had created to restore jobs in the dir.conf 
for two
different computers and it turned out that bacula picked wrong client IDs and 
the
restores either hang og exit with error. Removed one of the restores and am now 
able to
perform the restores correctly.

-- 
Erik P. Olsen - Copenhagen, Denmark
Fedora 31/64 bit Linux xfce Claws-Mail POP3 Gramps 5.1.2 Bacula 9.4.4


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Issue with concurrent jobs in disk based auto changer

[Josh Fisher]

On 4/4/2020 1:50 PM, Shaligram Bhagat, Yateen (Nokia - IN/Bangalore) wrote:


Hello,

...

All these jobs including the ones that are initially shown as 
 “created not yet running” eventually complete successfully, but after 
a long time (~36 Hours),


But the very purpose of concurrency is defeated.



You may also be looking at a network bottleneck. Is the file server on 
the same network as the clients? If so, then client data is traversing 
the same network twice and it would be better to run bacula-sd on the 
file server.


Also, if the db is being accessed across the network, then turning on 
attribute spooling may help.



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Baculum question

[Per Qvindesland via Bacula-users]

Hi 

Many thanks, I got the API to work on both api 1 and api 2 and the web host is 
able to talk to them and both are added in to the web host.

I do apologies if this is stated clearly somewhere in the documentation and I 
have done a fair bit of googllingbut the only thing I can’t get my head around 
is how to assign hosts api 1 and api 2 to one user,

I created a user on the each API hosts in then OAuth2 section then in configure 
I added the host with the client id’s to the web server host ran the tests, but 
when adding a new user then I still can’t add more then one API host to each 
user, is there anything that I have missed out on? 

Regards
Per





> On 5 Apr 2020, at 09:00, Marcin Haba  wrote:
> 
> Hello Per,
> 
> Yes, your web server configuration is correct.
> 
> Same changing web server configuration is not sufficient. Did you
> switch Baculum API instance to use OAuth2? You can do it in Baculum
> API configuration wizard. After that you need to define OAuth2 client
> account and connect Baculum Web (in Settings wizard) to Baculum API
> with these new OAuth2 settings.
> 
> I hope it helps.
> 
> Best regards,
> Marcin Haba (gani)
> 
> On Fri, 3 Apr 2020 at 17:30, Per Qvindesland via Bacula-users
>  wrote:
>> 
>> Hi
>> 
>> Thanks I am trying that but I am getting error Problem with authentication 
>> to Baculum API.
>> 
>> 
>> Is this the correct apache config to disable basic authentication?
>> 
>> #   
>>
>>AuthType Basic
>>AuthName "Baculum Auth"
>>AuthUserFile 
>> /usr/share/baculum/htdocs/protected/API/Config/baculum.users
>>Require valid-user
>>
>> #   
>> 
>> Regards
>> Per
>> 
>> 
>> 
>> 
>>> On 3 Apr 2020, at 14:57, Heitor Faria  wrote:
>>> 
 Hi
>>> 
>>> Hello Per,
>>> 
 Not sure if this is the correct place to ask but I am using Baculum and I 
 am
 wondering if it’s possible to configure baculum to allow users to view 
 multiple
 api hosts?
>>> 
>>> Yes 
>>> .
 
 Regards
>>> 
>>> Regards,
>>> 
 Per
 
 
 
 
 
 ___
 Bacula-users mailing list
 Bacula-users@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bacula-users
>>> 
>>> --
>>> MSc Heitor Faria
>>> CEO Bacula LATAM
>>> mobile1: + 1 909 655-8971
>>> mobile2: + 55 61 98268-4220
>>> [ https://www.linkedin.com/in/msc-heitor-faria-5ba51b3 ]
>>>  [ http://www.bacula.com.br/ ]
>>> 
>>> América Latina
>>> [ http://bacula.lat/ | bacula.lat ] | [ http://www.bacula.com.br/ | 
>>> bacula.com.br ]
>> 
>> 
>> 
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
> 
> 
> -- 
> "Greater love hath no man than this, that a man lay down his life for
> his friends." Jesus Christ
> 
> "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie
> za przyjaciół swoich." Jezus Chrystus



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] waiting on max Job jobs - not cancelled?

[Bernie Elbourn]

On 03/04/2020 13:19, Gary R. Schmidt wrote:


And the reason it doesn't get cancelled is because they are *not* identical jobs, one is "Differential" and the other 
is "Incremental."


You need to run the Differential, Incremental, and Full jobs at different priorities - that reflect how you want 
things done - and set "Cancel Lower Level Duplicate = Yes" and "Allow Mixed Priority = Yes" in them all.


Hi,

Its an intriguing thought but the bacula 7 reference manual only mentions the 
name...

"Allow Duplicate Jobs =  A duplicate job in the sense we use it here 
means a second or sub-
sequent job with the same name starts. This happens most frequently when the 
first job runs longer
than expected because no tapes are available."

... it then goes on to explain options for lower or higher jobs without 
reference to priority...

"Cancel Lower Level Duplicates =  If Allow Duplicate Jobs is set to no 
and this directive
is set to yes, Bacula will choose between duplicated jobs the one with the 
highest level. For example,
it will cancel a previous Incremental to run a Full backup. It works only for 
Backup jobs. The default
is no. If the levels of the duplicated jobs are the same, nothing is done and 
the other Cancel XXX
Duplicate directives will be examined."

I am pretty sure that the configuration files are ok. I have now copied the configuration to another system, and on that 
setup lower level duplicates are cancelled as expected rather than queued "max job jobs".


Have to say debugging this peculiarity is baffling me...

Rgs

Bernie



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users