Re: [Bacula-users] Copy backups to more than one storage

[Heitor Faria]

Hello All,

> I am using PoolUncopiedJobs with a RunAfterJob which executes a script
> when the job successfully finishes. The script inserts the job ID of
> a copied job into a table created specifically for that purpose.
> The original job ID can be passed to a shell script using "%I".
> 
> This requires additional admin job that starts jobs that will perform
> a copy using the list of job IDs from the table.
> Those copy jobs also use RunAfterJob calling a shell script that
> removes a job ID from the table in case the copy job finished
> successfully.

This is the Selection Pattern I use:

#
# Copies all jobs never copied which can be used for more than one destination 
pool

SELECT public.job.jobid FROM public.pool INNER JOIN public.job ON 
public.pool.poolid=public.job.poolid WHERE public.pool.name='' 
AND Job.jobBytes > 0 AND public.job.jobid NOT IN (SELECT public.job.priorjobid 
FROM public.pool INNER JOIN public.job ON public.pool.poolid=public.job.poolid 
WHERE public.pool.name='');
#

Rgds.
-- 
MSc Heitor Faria (Miami/USA) 
Bacula LATAM CIO 

mobile1: + 1 909 655-8971 
mobile2: + 55 61 98268-4220 
[ https://www.linkedin.com/in/msc-heitor-faria-5ba51b3 ][ 
http://www.bacula.com.br/ ] 
[ http://bacula.lat/ | bacula.lat ] | [ http://www.bacula.com.br/ | 
bacula.com.br ]


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Copy backups to more than one storage

[Josip Deanovic]

On 2023-01-17 17:45, Bill Arlofski via Bacula-users wrote:

On 1/17/23 06:05, Ivan Villalba via Bacula-users wrote:


How can I run two differnet copy job that copies the same jobid with 
the PoolUncopiedJobs ?


You can't.

The PoolUncopiedJobs does exactly what its name suggests: It copies
jobs in a pool that have not been copied to some other pool.

If you want to copy the same backup jobs to more than one other pool,
you will need to use `Selection type = SQLQuery` and
then use an appropriate SQL query for the `SelectionPattern` to
generate a list of JobIds to run the second set of copies.



I am using PoolUncopiedJobs with a RunAfterJob which executes a script
when the job successfully finishes. The script inserts the job ID of
a copied job into a table created specifically for that purpose.
The original job ID can be passed to a shell script using "%I".

This requires additional admin job that starts jobs that will perform
a copy using the list of job IDs from the table.
Those copy jobs also use RunAfterJob calling a shell script that
removes a job ID from the table in case the copy job finished
successfully.

That way it is possible to copy jobs as many time as needed.
It is possible to use different pools which is useful in case one
need to maintain several pools which contain all the jobs stored on
different media which also rotate several times per week.

I like the level of flexibility Bacula offers.

It is worth mentioning that copy job builds a list of jobs that
need to be copied at the time of job execution.


Regards

--
Josip Deanovic


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Copy backups to more than one storage

[Phil Stracchino]

On 1/17/23 11:45, Bill Arlofski via Bacula-users wrote:

On 1/17/23 06:05, Ivan Villalba via Bacula-users wrote:
  >

How can I run two differnet copy job that copies the same jobid with the 
PoolUncopiedJobs ?


You can't.

The PoolUncopiedJobs does exactly what its name suggests: It copies jobs in a 
pool that have not been copied to some other pool.

If you want to copy the same backup jobs to more than one other pool, you will 
need to use `Selection type = SQLQuery` and
then use an appropriate SQL query for the `SelectionPattern` to generate a list 
of JobIds to run the second set of copies.



Which, I note, is what I do ANYWAY, just in case something goes wrong 
and I have to *rerun* the copy job.



--
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Copy backups to more than one storage

[Ivan Villalba via Bacula-users]

Thanks Bill ,

I'm going to try something proposed in this paper:
https://bacula.org/whitepapers/ObjectStorage.pdf

So the idea is to upload the volume data on /bacula-storage/ to aws s3
using aws cli. I'm doing this as a workaround on the s3+objectlock issue,
and this solves the problem of the two copies of one job as well.

Two solutions with one shot.

Hope this helps.

Thank you all !

On Tue, Jan 17, 2023 at 5:47 PM Bill Arlofski via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

> On 1/17/23 06:05, Ivan Villalba via Bacula-users wrote:
>  >
> > How can I run two differnet copy job that copies the same jobid with the
> PoolUncopiedJobs ?
>
> You can't.
>
> The PoolUncopiedJobs does exactly what its name suggests: It copies jobs
> in a pool that have not been copied to some other pool.
>
> If you want to copy the same backup jobs to more than one other pool, you
> will need to use `Selection type = SQLQuery` and
> then use an appropriate SQL query for the `SelectionPattern` to generate a
> list of JobIds to run the second set of copies.
>
>
> Hope this helps.
> Bill
>
> --
> Bill Arlofski
> w...@protonmail.com
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Copy backups to more than one storage

[Bill Arlofski via Bacula-users]

On 1/17/23 06:05, Ivan Villalba via Bacula-users wrote:
>

How can I run two differnet copy job that copies the same jobid with the 
PoolUncopiedJobs ?


You can't.

The PoolUncopiedJobs does exactly what its name suggests: It copies jobs in a 
pool that have not been copied to some other pool.

If you want to copy the same backup jobs to more than one other pool, you will 
need to use `Selection type = SQLQuery` and
then use an appropriate SQL query for the `SelectionPattern` to generate a list 
of JobIds to run the second set of copies.


Hope this helps.
Bill

--
Bill Arlofski
w...@protonmail.com



signature.asc
Description: OpenPGP digital signature
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Copy backups to more than one storage

[Ivan Villalba via Bacula-users]

I have confirmed this behaviour:

The PoolUncopiedJobs select type for Copy jobs, determines that if the
second copy job will return not jobIDs to copy.

1) Backup job (Backup on main backup server's bacula SD), does a backup.
2) First copy job (Copy to 2nd backup server's bacula SD), does the copy of
the last backup with the PoolUncopiedJobs
3) Second copy job  (Copy to S3 with objectlock) will not found jobs to
copy as per the first copy job already copied the last backup.

Anyone had to deal with this situation? How can I run two differnet copy
job that copies the same jobid with the PoolUncopiedJobs ?

Thanks.

On Tue, Nov 29, 2022 at 4:19 PM Ivan Villalba 
wrote:

> Hi there,
>
> In order to follow the 3-2-1 backups strategy, I need to create a second
> copy type job to send backups to s3. The current client definition have two
> jobs, one for the main backup (bacula server), and a copy type job that,
> using the Next Pool directive in the original Pool, sends the backups to an
> external SD in a secondary bacula server.
>
> How do I create a second copy type job but using a third pool (not the
> original Next Pool), so I can send backups to S3?
>
>
> Thanks in advance.
>
> --
> Ivan Villalba
> SysOps - Marfeel
>
>
>

-- 
Ivan Villalba
SysOps




[image: Inline images 4]

 [image: Inline images 3]



Avda. Josep Tarradellas 20-30, 4th Floor

08029 Barcelona, Spain

ES: (34) 93 178 59 50
<%2834%29%2093%20178%2059%2050%20%C2%A0ext.%20107>
US: (1) 917-341-2540 <%281%29%20917-341-2540%20ext.%20107>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Copy backups to more than one storage

[Bill Arlofski via Bacula-users]

On 11/29/22 08:19, Ivan Villalba via Bacula-users wrote:

Hi there,

In order to follow the 3-2-1 backups strategy, I need to create a second copy type job to send backups to s3. The current 
client definition have two jobs, one for the main backup (bacula server), and a copy type job that, using the Next Pool 
directive in the original Pool, sends the backups to an external SD in a secondary bacula server.


How do I create a second copy type job but using a third pool (not the original 
Next Pool), so I can send backups to S3?


Thanks in advance.

--
Ivan Villalba
SysOps - Marfeel



Hello Ivan,

A Pool's NextPool setting may be overridden in a Job, or in a Schedule.


Hope this helps!
Bill

--
Bill Arlofski
w...@protonmail.com



signature.asc
Description: OpenPGP digital signature
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Copy backups to more than one storage

[Ivan Villalba via Bacula-users]

Hi there,

In order to follow the 3-2-1 backups strategy, I need to create a second
copy type job to send backups to s3. The current client definition have two
jobs, one for the main backup (bacula server), and a copy type job that,
using the Next Pool directive in the original Pool, sends the backups to an
external SD in a secondary bacula server.

How do I create a second copy type job but using a third pool (not the
original Next Pool), so I can send backups to S3?


Thanks in advance.

-- 
Ivan Villalba
SysOps - Marfeel
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users