Re: [Bacula-users] Virtual tapes or virtual disks

[Josip Deanovic]

On 23.01.2022 11:37, Radosław Korzeniewski wrote:

Hello,

pt., 21 sty 2022 o 14:22 Peter Milesson via Bacula-users 
 napisał(a):


If somebody has got experience with disk based, multi volume Bacula
backup, I would be grateful about some information (tips, what to
expect, pitfalls, etc.).


The best IMVHO (but not the only mine) is to configure one job = one 
volume. You will get no real benefit to limit the size of a single 
volume.
In the single volume = single job configuration you can set up job 
retention very easily as purging a volume will purge a single job only.
It is not required to "wait" a particular volume to fill up to start 
retention. Purging a volume affects a single job only. And finally you 
end up with a way less number of volumes then when limiting its size to 
i.e. 10G.



There are many different approaches which can fit different 
requirements.

I don't see the benefit of having a single job per volume as Bacula
is tracking media, files, jobs and everything else.
That's why Bacula has a catalog which allows the backup system
to determine the location and state of volumes, jobs, files, etc.

To logically separate backup data I use pools and leave the rest
to Bacula.

When Bacula needs a particular file volume, if it's available Bacula
will simply use it and if it's not or if we are using tape volume
which is currently not in the tape drive/library, Bacula will ask
for the volume by name.

The number of smaller file volumes (e.g. 10GB) is not an issue as
Bacula is handling them correctly and automatically (provided that
Bacula is correctly configured, of course).


I'll go through few examples where smaller file volumes (e.g. 10GB)
could prove useful:

1. If the catalog database get corrupted or completely lost,
   due to the the small size, it's easier and faster to handle
   and determine volumes which contain database backup.
   That makes the process of importing the data into a new
   catalog database using a tool such as bscan easier.

2. Similar to 1), it is easier to manage small file volumes and
   extract particular jobs from a volume using bextract tool.

3. If the space is an issue (as it usually is), bigger volumes
   tend to eat more space which cannot be reused (volume
   cannot be recycled) as long as the volume contains a single
   job we want to preserve.

4. Although I don't like that approach, sometimes people chose
   to sync or copy whole file volumes to a secondary location
   using the usual tools such as rsync, cp and similar.
   In such case it is better to keep file volumes small.

5. When recycling a file volume, it will take longer time to
   wipe bigger file volume. If a volume is smaller it will
   take less time to recycle ensuring more time windows where
   other tasks could benefit from I/O performance. In case of
   large file volumes all other tasks would have to fight for
   the opportunity to access the file system and that gets more
   obvious when a slow network file system is being used.

6. In case of any kind of corruption of a file volume due to
   the file system corruption or damage in transport, it is
   likely that less data will be lost in case of a smaller
   file volume. And again, it's easier to handle smaller file
   volume when trying to recover pieces of data.


Regards!

--
Josip Deanovic


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Virtual tapes or virtual disks

[Peter Milesson via Bacula-users]

On 25.01.2022 9:31, Lionel PLASSE wrote:

I’ve made a rotation system like this

With one Bacula storage “backup” corresponding one Bacula device 
“Backup”<=> root directory /pool-bacula/ automounted by udev.

And 3 pool full, diff, incr attached to this device

I have 8 usb disk xfs formatted with a xfs label “Bacula” (better than 
uuid to configure udev rules) so udev can mount it automatically when 
plugged to the Debian system. I don't use Bacula for usb disk mounting 
task


On each disk I label one volume:
3 for each 3 month full rotation volumes ,
Four for each 4 weeks diff rotation
one for the daily incremental auto purged each week

By configuring the good retention period between week days and month 
and by adding the correct number on jobs volume on each one, you can 
easily configure a schedule with 3 steps ,
one for full attached to full pool on each first Wednesday a month for 
example,
one for diff attached to diff pool each week from 2nd Wednesday to 5th 
Wednesday

and the third for incremental each day from Monday to Thursday.

By this schedule you can keep a great number of backups so each day 
you can restore the previous until the Monday with incremental 
backups, each week kept by the differental backups and the 3 full 
montly too.


De : Radosław Korzeniewski 
Envoyé : dimanche 23 janvier 2022 11:37
À : Peter Milesson 
Cc : bacula-users@lists.sourceforge.net
Objet : Re: [Bacula-users] Virtual tapes or virtual disks

Hello,

pt., 21 sty 2022 o 14:22 Peter Milesson via Bacula-users 
 napisał(a):

If somebody has got experience with disk based, multi volume Bacula
backup, I would be grateful about some information (tips, what to
expect, pitfalls, etc.).

The best IMVHO (but not the only mine) is to configure one job = one 
volume. You will get no real benefit to limit the size of a single volume.
In the single volume = single job configuration you can set up job 
retention very easily as purging a volume will purge a single job only.
It is not required to "wait" a particular volume to fill up to start 
retention. Purging a volume affects a single job only. And finally you 
end up with a way less number of volumes then when limiting its size 
to i.e. 10G.


best regards
--
Radosław Korzeniewski
mailto:rados...@korzeniewski.net

Hi Lionel,

thanks for the information. I've tried your way of doing backups after 
throwing out the physical tape drive. It just wasn't workable. The first 
thing is having office personnel properly managing the different USB 
drives (same problem as with tapes). Second, backup to USB drives takes 
forever, and there is not time enough for a full backup to complete 
comes Monday morning, not to speak of the few times there were backup 
glitches and the necessity to repeat the full backup. So the only 
solution for me is backup to a huge fixed storage. Now I'm planning how 
to implement volume management on this fixed storage.


Best regards,

Peter




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Virtual tapes or virtual disks

[Peter Milesson via Bacula-users]

On 23.01.2022 11:37, Radosław Korzeniewski wrote:

Hello,

pt., 21 sty 2022 o 14:22 Peter Milesson via Bacula-users 
 napisał(a):


If somebody has got experience with disk based, multi volume Bacula
backup, I would be grateful about some information (tips, what to
expect, pitfalls, etc.).


The best IMVHO (but not the only mine) is to configure one job = one 
volume. You will get no real benefit to limit the size of a single volume.
In the single volume = single job configuration you can set up job 
retention very easily as purging a volume will purge a single job only.
It is not required to "wait" a particular volume to fill up to start 
retention. Purging a volume affects a single job only. And finally you 
end up with a way less number of volumes then when limiting its size 
to i.e. 10G.


best regards
--
Radosław Korzeniewski
rados...@korzeniewski.net


Hi Radosław,

thanks for your input. I'm still in the planning phase for the new 
backup server, so all information is valuable.


Excuse me some elementary questions. How do you define volumes in the 
pool for your configuration? I've been using Bacula a little more than 
11 years, but up till now I've used virtual tapes with fixed sizes. Once 
setup, it's just ticking, and the original configuration stays.


I use a one year retention scheme, and that will not change. I have got 
no reason to purge jobs within that period and after a year, I would 
like to do as little work as possible, having old volumes purged 
automatically, when there is no more room on the RAID array. Using 
tapes, this was automatic. When all volumes were full, the volume with 
the oldest write time is purged, and reused.


Best regards,

Peter
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Virtual tapes or virtual disks

[Lionel PLASSE]

Yes, one volume, one disk, plus all the Bacula conf files and the bootstrap bsr 
file


PLASSE Lionel | Administrateur systŠmes et r‚seaux 
221 All‚e de F‚tan
01600 TREVOUX | Plan d'accŠs
Tel : 04.37.49.91.39
pla...@cofiem.fr
www.cofiem.fr | www.cofiem-robotics.fr

 



-Message d'origine-
De : Josh Fisher  
Envoyé : mardi 25 janvier 2022 15:10
À : Lionel PLASSE ; bacula-users@lists.sourceforge.net
Objet : Re: [Bacula-users] Virtual tapes or virtual disks


On 1/25/22 03:31, Lionel PLASSE wrote:
> I’ve made a rotation system like this
>
>   With one Bacula storage “backup” corresponding one Bacula device 
> “Backup”<=> root directory /pool-bacula/  automounted by udev.
> And 3 pool full, diff, incr attached to this device
>
> I have 8 usb disk xfs formatted with a xfs label “Bacula” (better than 
> uuid to configure udev rules) so udev can mount it automatically when 
> plugged to the Debian system. I don't use Bacula for usb disk mounting 
> task
>
> On each disk I label one volume:
>   3 for each 3 month full rotation volumes , Four for each 4 weeks 
> diff rotation one for the daily incremental auto purged each week


To be clear, you are using only a single volume per USB disk?


>
>   By configuring the good retention period between week days and month  and 
> by adding the correct number on jobs volume  on each one, you can easily 
> configure a schedule with 3 steps ,
> one for full attached to full pool on each first Wednesday a month  for 
> example,
>   one for diff attached to diff pool each week from 2nd Wednesday to 5th 
> Wednesday
> and the third for incremental each day from Monday to Thursday.
>
> By this schedule you can keep a great number of backups so  each day you can 
> restore the previous until the Monday with  incremental backups, each week 
> kept by the differental backups and the 3 full montly too.
>
> De : Radosław Korzeniewski 
> Envoyé : dimanche 23 janvier 2022 11:37
> À : Peter Milesson 
> Cc : bacula-users@lists.sourceforge.net
> Objet : Re: [Bacula-users] Virtual tapes or virtual disks
>
> Hello,
>
> pt., 21 sty 2022 o 14:22 Peter Milesson via Bacula-users 
>  napisał(a):
> If somebody has got experience with disk based, multi volume Bacula
> backup, I would be grateful about some information (tips, what to
> expect, pitfalls, etc.).
>
> The best IMVHO (but not the only mine) is to configure one job = one volume. 
> You will get no real benefit to limit the size of a single volume.
> In the single volume = single job configuration you can set up job retention 
> very easily as purging a volume will purge a single job only.
> It is not required to "wait" a particular volume to fill up to start 
> retention. Purging a volume affects a single job only. And finally you end up 
> with a way less number of volumes then when limiting its size to i.e. 10G.
>
> best regards
> --
> Radosław Korzeniewski
> mailto:rados...@korzeniewski.net
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Virtual tapes or virtual disks

[Josh Fisher]

On 1/25/22 03:31, Lionel PLASSE wrote:

I’ve made a rotation system like this

  With one Bacula storage “backup” corresponding one Bacula device “Backup”<=> 
root directory /pool-bacula/  automounted by udev.
And 3 pool full, diff, incr attached to this device

I have 8 usb disk xfs formatted with a xfs label “Bacula” (better than uuid to 
configure udev rules) so udev can mount it automatically when plugged to the 
Debian system. I don't use Bacula for usb disk mounting task

On each disk I label one volume:
  3 for each 3 month full rotation volumes ,
Four for each 4 weeks diff rotation
one for the daily incremental auto purged each week



To be clear, you are using only a single volume per USB disk?




  By configuring the good retention period between week days and month  and by 
adding the correct number on jobs volume  on each one, you can easily configure 
a schedule with 3 steps ,
one for full attached to full pool on each first Wednesday a month  for example,
  one for diff attached to diff pool each week from 2nd Wednesday to 5th 
Wednesday
and the third for incremental each day from Monday to Thursday.

By this schedule you can keep a great number of backups so  each day you can 
restore the previous until the Monday with  incremental backups, each week kept 
by the differental backups and the 3 full montly too.

De : Radosław Korzeniewski 
Envoyé : dimanche 23 janvier 2022 11:37
À : Peter Milesson 
Cc : bacula-users@lists.sourceforge.net
Objet : Re: [Bacula-users] Virtual tapes or virtual disks

Hello,

pt., 21 sty 2022 o 14:22 Peter Milesson via Bacula-users 
 napisał(a):
If somebody has got experience with disk based, multi volume Bacula
backup, I would be grateful about some information (tips, what to
expect, pitfalls, etc.).

The best IMVHO (but not the only mine) is to configure one job = one volume. 
You will get no real benefit to limit the size of a single volume.
In the single volume = single job configuration you can set up job retention 
very easily as purging a volume will purge a single job only.
It is not required to "wait" a particular volume to fill up to start retention. 
Purging a volume affects a single job only. And finally you end up with a way less number 
of volumes then when limiting its size to i.e. 10G.

best regards
--
Radosław Korzeniewski
mailto:rados...@korzeniewski.net

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Zapisywanie kopii jednocześnie na dwóch nośnikach

[Radosław Korzeniewski]

Hello,

wt., 25 sty 2022 o 11:41 Tomasz Świderski  napisał(a):

> Is it possible to configure one job to save on two media at the same
> time, e.g. on HDD and tape drive?


No. Currently not. I'm pretty sure Bacula Systems is working on such a
feature for Bacula Enterprise. I have no info what is a timeline for this.


> These devices are in the same storage
> but in different pools
>

But you can use a volume/job cloning feature which is an asynchronous job
executed after the main backup job. In this case you can "duplicate" data
on different devices of the same or remote storage.

best regards
-- 
Radosław Korzeniewski
rados...@korzeniewski.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Zapisywanie kopii jednocześnie na dwóch nośnikach

[Tomasz Świderski]

Is it possible to configure one job to save on two media at the same 
time, e.g. on HDD and tape drive? These devices are in the same storage 
but in different pools




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Virtual tapes or virtual disks

[Lionel PLASSE]

I’ve made a rotation system like this 

 With one Bacula storage “backup” corresponding one Bacula device “Backup”<=> 
root directory /pool-bacula/  automounted by udev. 
And 3 pool full, diff, incr attached to this device

I have 8 usb disk xfs formatted with a xfs label “Bacula” (better than uuid to 
configure udev rules) so udev can mount it automatically when plugged to the 
Debian system. I don't use Bacula for usb disk mounting task

On each disk I label one volume:
 3 for each 3 month full rotation volumes ,
Four for each 4 weeks diff rotation 
one for the daily incremental auto purged each week

 By configuring the good retention period between week days and month  and by 
adding the correct number on jobs volume  on each one, you can easily configure 
a schedule with 3 steps ,
one for full attached to full pool on each first Wednesday a month  for example,
 one for diff attached to diff pool each week from 2nd Wednesday to 5th 
Wednesday 
and the third for incremental each day from Monday to Thursday. 

By this schedule you can keep a great number of backups so  each day you can 
restore the previous until the Monday with  incremental backups, each week kept 
by the differental backups and the 3 full montly too.

De : Radosław Korzeniewski  
Envoyé : dimanche 23 janvier 2022 11:37
À : Peter Milesson 
Cc : bacula-users@lists.sourceforge.net
Objet : Re: [Bacula-users] Virtual tapes or virtual disks

Hello,

pt., 21 sty 2022 o 14:22 Peter Milesson via Bacula-users 
 napisał(a):
If somebody has got experience with disk based, multi volume Bacula 
backup, I would be grateful about some information (tips, what to 
expect, pitfalls, etc.).

The best IMVHO (but not the only mine) is to configure one job = one volume. 
You will get no real benefit to limit the size of a single volume.
In the single volume = single job configuration you can set up job retention 
very easily as purging a volume will purge a single job only.
It is not required to "wait" a particular volume to fill up to start retention. 
Purging a volume affects a single job only. And finally you end up with a way 
less number of volumes then when limiting its size to i.e. 10G.

best regards
--
Radosław Korzeniewski
mailto:rados...@korzeniewski.net

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users