Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Marco Gaiarin]

Mandi! Marco Gaiarin
  In chel di` si favelave...

> OK, i have to truncate them, not delete, but... i've understood, i think...

Script start to work. I'll post here more after some cleanup... but i still
hit strange things in bacula. ;-)

*purge yes volume=VWSSV1_0020 action=truncate storage=SVPVE3FileMulti
Automatically selected Catalog: BaculaLNF
Using Catalog "BaculaLNF"
Using Catalog "BaculaLNF"
Connecting to Storage daemon SVPVE3FileMulti at svpve3.sv.lnf.it:9103 ...
Truncate Volume "VWSSV1_0020" skipped. Status is "Used", but must be "Purged".
You have messages.

*purge yes volume=VWSSV1_0020 storage=SVPVE3FileMulti
This command can be DANGEROUS!!!
It purges (deletes) all Files from a Job,
JobId, Client or Volume; or it purges (deletes)
all Jobs from a Client or Volume without regard
to retention periods. Normally you should use the
PRUNE command, which respects retention periods.
Using Catalog "BaculaLNF"
There are no more Jobs associated with Volume "VWSSV1_0020". Marking it purged.
*purge yes volume=VWSSV1_0020 action=truncate storage=SVPVE3FileMulti
Using Catalog "BaculaLNF"
Connecting to Storage daemon SVPVE3FileMulti at svpve3.sv.lnf.it:9103 ...
3000 OK label. VolBytes=235 VolABytes=0 VolType=1 Volume="VWSSV1_0020" 
Device="MultiFileStorage0" (/rpool-backup/bacula)
The volume "VWSSV1_0020" has been truncated

Clearly i have in pool 'Action on Purge = Truncate', but still i have to
purge the media first, then truncate them (with 'purge ... action=truncate
...' or directly with 'truncate').


Why?! Thanks.

-- 




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Bill Arlofski via Bacula-users]

On 10/2/24 10:00 AM, Marco Gaiarin wrote:

Mandi! Gary R. Schmidt
   In chel di` si favelave...


RTFM again, it took me about three goes to understand it - and I've been
doing backups since 9-track tape drives were vertical!


Mmmm... i've found references to scratch pool but really i've nefer
understood; if i look in docs i found a rather laconic:


https://www.bacula.org/9.4.x-manuals/en/main/Configuring_Director.html#SECTION0020161000

seems there's no whitepaper on scratch pools...


So, RTFM where?! ;-)


Hello Marco,

Typically Scratch pools are not too useful for disk volumes.

The idea behind a Scratch pool is the following:

Let's say you have two pools: "Full", and "Inc"

If you do not have a "ScratchPool" to pull from, and a RecyclePool to send recycled volumes back into, then tapes created in 
the Inc pool (for example) will never be available to be used in the Full pool - even if they are all past their
retention 
and have all been pruned and recycled. They are doomed to this Inc pool forever. :)


So, you could have job(s) writing to the Full pool all waiting on media which is technically available (pruned and living in 
the Inc pool), but not available for use by jobs in the Full pool.


So, what you do is set "ScratchPool = ScratchPoolName" and "RecyclePool = ScratchPoolName" in your Full and Inc Pools. Then, 
make sure to enable Recycling (Recycle = yes) in the pools.


Then, when tapes are labeled (using the 'label barcodes' command in bconsole), you specify the ScratchPoolName as the initial 
pool where they will be created and then Bacula will pull from the Scratch pool when a tape is needed, and put it back there 
when it is recycled so that the tape is then free to move between the Full and Inc pool as it becomes available, and as 
necessary by Bacula's needs.



I do have a use case for ScratchPool and RecyclePool settings in Disk volume 
pools... In my case I use
Josh Fisher's 
'vchanger' so each of my xTB removable drives can have 10 file volumes in either my Inc or Full offsite pools. Disk volumes 
on each physical/removable xTB disk can freely move as described above so I never run into a situation where the whole 
physical disk is full of 10GB volumes in the full pool when Inc pool volumes are needed for a job and vice versa.



Note: A pool named "Scratch" is treated a bit special by Bacula. If no `ScratchPool` is specified in a pool, Bacula will look 
for volumes with the correct MediaType in this pool.   But, you can name your scratch pools anything you like and never use 
this specific one.


For example, If an SD manages more than one tape library, you can have a set of 
pools for Lib1:
 - Lib1_Scratch, Lib1_Full, Lib1_Inc

And then a set of pools for Lib2:
 - Lib2_Scratch, Lib2_Full, Lib2_Inc


Hope this helps,
Bill

--
Bill Arlofski
w...@protonmail.com



signature.asc
Description: OpenPGP digital signature
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Marco Gaiarin]

Mandi! Gary R. Schmidt
  In chel di` si favelave...

> RTFM again, it took me about three goes to understand it - and I've been 
> doing backups since 9-track tape drives were vertical!

Mmmm... i've found references to scratch pool but really i've nefer
understood; if i look in docs i found a rather laconic:


https://www.bacula.org/9.4.x-manuals/en/main/Configuring_Director.html#SECTION0020161000

seems there's no whitepaper on scratch pools...


So, RTFM where?! ;-)

-- 




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Gary R. Schmidt]

On 01/10/2024 00:26, Marco Gaiarin wrote:

Mandi! Phil Stracchino
   In chel di` si favelave...


Incidentally, the way I solved this was:
— Use volumes ONCE
— Prune expired volumes into the scratch pool
— Regularly clean and delete all volumes found in the scratch pool (or,
as an added later refinement just in case of mishaps, any volume not
found in ANY pool)


OK, i suppose you have the (main) pool where the full(s) reside, and the
scratch pool used for incrementals.


No, the Incremental pool is for Incremental Backups.

The Scratch pool is for blank, empty, or recycled volumes, it is where 
Bacula goes looking for tapes/disks/whatever when it fills up what it is 
using.


RTFM again, it took me about three goes to understand it - and I've been 
doing backups since 9-track tape drives were vertical!


Cheers,
GaryB-)


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Marco Gaiarin]

Mandi! Chris Wilkinson
  In chel di` si favelave...

> Sure. I put it here. It's a bash script. will need a bit of tweaking for
> your particular setup.
> https://drive.google.com/file/d/1WOFIjgZMx-XL2Jv7FqGupnjctLpUEhLG/view?usp=drivesdk

OK, mostly understood. Bright idea!

But for media purging you use 'query 14': do you mean 'standard' query 14,
so 'List Jobs stored for a given Volume name'?


So the script build a list of volumes, cycle throug volume checking if they
contain jobs, and if not, delete them.


OK, i have to truncate them, not delete, but... i've understood, i think...


Thanks.

-- 




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Marco Gaiarin]

Mandi! Phil Stracchino
  In chel di` si favelave...

> Incidentally, the way I solved this was:
> — Use volumes ONCE
> — Prune expired volumes into the scratch pool
> — Regularly clean and delete all volumes found in the scratch pool (or, 
> as an added later refinement just in case of mishaps, any volume not 
> found in ANY pool)

OK, i suppose you have the (main) pool where the full(s) reside, and the
scratch pool used for incrementals.

But when you consolidate backups, you need at least TWO volumes on main
pool, right? One for reading the old full, one to write the new one.

So still you have a full volume to purge/truncate after a successful
consolidation...

-- 




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Phil Stracchino]

On 9/28/24 15:20, Chris Wilkinson wrote:
Sure. I put it here. It's a bash script. will need a bit of tweaking for 
your particular setup.


https://drive.google.com/file/d/1WOFIjgZMx-XL2Jv7FqGupnjctLpUEhLG/view?usp=drivesdk 




Incidentally, the way I solved this was:
— Use volumes ONCE
— Prune expired volumes into the scratch pool
— Regularly clean and delete all volumes found in the scratch pool (or, 
as an added later refinement just in case of mishaps, any volume not 
found in ANY pool)



--
  Phil Stracchino
  Fenian House Publishing
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Chris Wilkinson]

Sure. I put it here. It's a bash script. will need a bit of tweaking for
your particular setup.

https://drive.google.com/file/d/1WOFIjgZMx-XL2Jv7FqGupnjctLpUEhLG/view?usp=drivesdk

-Chris-

On Sat, 28 Sept 2024, 19:52 Dan Langille,  wrote:

> On Fri, Sep 27, 2024, at 3:50 PM, Chris Wilkinson wrote:
>
> I have a script to do that and a few other things. I use disc based
> backups and am always conscious of running out of space. I'd be happy to PM
> it to you if you wish. It's too big to send via the list.
>
>
> Would you consider just publishing the script, thereby making it available
> to everyone?
>
> Just tossing it in gist.github.com, for example, would help others.
>
> --
>   Dan Langille
>   d...@langille.org
>
>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Dan Langille]

On Fri, Sep 27, 2024, at 3:50 PM, Chris Wilkinson wrote:
> I have a script to do that and a few other things. I use disc based backups 
> and am always conscious of running out of space. I'd be happy to PM it to you 
> if you wish. It's too big to send via the list.

Would you consider just publishing the script, thereby making it available to 
everyone?

Just tossing it in gist.github.com, for example, would help others.

--
  Dan Langille
  d...@langille.org

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Aggressively prune (and truncate) file volumes on Progressive backups...

[Chris Wilkinson]

I have a script to do that and a few other things. I use disc based backups
and am always conscious of running out of space. I'd be happy to PM it to
you if you wish. It's too big to send via the list.

-Chris Wilkinson

On Fri, 27 Sep 2024, 20:12 Marco Gaiarin,  wrote:

>
> I've setup a set of jobs on windows client machine, as progressive virtual
> full,
> eg infinite retention and cancelling job via consolidation.
>
> I've on the pool:
>
> Action On Purge = Truncate
>
> but jobs get deleted, and volumes get purged (and truncated) only if
> needed.
>
>
> There's some way, after the consolidation, to purge (and truncate) volume
> that contain no jobs?
>
>
> Thanks.
>
> --
>
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users