Hello Nick,

Bill, I'm here :-) Sorry for not jumping in earlier.

Nick, I will try to summarize a bit about retention values and
pruning/truncation in Bacula. Pruning is about "the deletion of jobs and
files from Catalog". Automatic or manual prune will not touch the data in
the volumes, but only the jobs and files records in the Catalog are deleted.

The easiest way is to allow Bacula to automatically prune Jobs and Files
from the Catalog. It means to have "AutoPrune = Yes" in both the Client
resource and in the Pool resource.

1) having "AutoPrune = Yes" in the Client resource -> at the end of each
job run associated to the Client/Pool of the job, Bacula will check if the
retention have passed for all the jobids associated to the Client/Pool and
the jobs and files are deleted from the Catalog if the retention has
passed. You will see this kind of message:

| bacula-dir JobId 5104: Begin pruning Jobs older than 6 months .         |
| bacula-dir JobId 5104: No Jobs found to prune.                          |
| bacula-dir JobId 5104: Begin pruning Files.                             |
| bacula-dir JobId 5104: No Files found to prune.                         |
| bacula-dir JobId 5104: End auto prune.                                  |

If any job/file has reached the retention based on the values defined in
the Client and/or Pool resources. The values set in a Pool resource will
take precedence over the values set in a Client resource.

2) having "AutoPrune = Yes" in the Pool resource -> when Bacula needs to
reuse a volume for a job (if Bacula can't create a new volume because the
max number of volumes in the pool is reached and there is no volume in
Append, Purged or Recycle status), the recycling algorithm will try to
prune jobs and files to reuse a volume. It will start searching for the
oldest volume in the pool. Then, Bacula will check if this volume has no
more jobs associated with it in the Catalog (it means the jobs and files
have been pruned so this volume can be reused). If so, you will see
messages like:

04-Nov 15:15 bacula-dir JobId 3560: There are no more Jobs associated with
Volume "Vol-0038". Marking it purged.
04-Nov 15:15 bacula-dir JobId 3560: All records pruned from Volume
"Vol-0038"; marking it "Purged"
04-Nov 15:15 bacula-dir JobId 3560: Recycled volume "Vol-0038"

This is a job log where Bacula couldn't create a new volume for jobid=3560
and it reused Vol-0038 because the jobs associated with it had been pruned
(by automatic prune after the job run) previously.

Now, to force the prune of volumes that have the VolumeRetention (value
defined in the Pool resource) expired, the command is:

* prune volume expired yes

This command is dangerous, it will delete all jobs and files associated
with any volume by comparing only the "VolumeRetention" of the volume. This
value is defined in the Pool resource, but it is specific for a volume and
Bacula will check the value in the "volretention" field in the Media table
in the Bacula database for each volume.

This is why when you change the "VolumeRetention" value in the Pool
resource the value of the already existing volumes is not modified and you
have to run:

* update volume fromallpools
All Volume defaults updated from "Default" Pool record.
All Volume defaults updated from "File" Pool record.
All Volume defaults updated from "Scratch" Pool record.

After that, the volretention values in the Catalog for the existing volumes
is updated to the value defined in the Pool configuration and the "prune
expired volume yes" will use the new value to compare with.

Then, to truncate volumes you must have the following configuration in the
Pool resource:

ActionOnPurge = Truncate

Please remember to reload the configuration in bconsole using the reload
command after adding the above line.

After that, you will be able to "truncate volumes in Purged status" using
the following command:

* truncate volume allpools storage=File1

Please let us know if it helps.


On Wed, Nov 23, 2022 at 11:04 PM Nick Bright <nbri...@kwikom.com> wrote:

> I changed the Volume retention to 7 days, repeated all of the commands,
> and ran a BackupCatalog job; still reports "The job needs media"
> On 11/23/22 13:39, Bill Arlofski via Bacula-users wrote:
> > On 11/23/22 12:27, Nick Bright wrote:
> >> In the Pool definition there is a Volume Retention of 365 days; however
> >> each client has it's own File Retention and Job Retention (7 days each)
> >>
> >> Shouldn't this result in the data contained within the volumes being
> >> expired, and thus the volume could be recycled/purged/truncated once
> >> its' containing datas' retention period has expired?
> >
> >
> >
> > A couple things to consider here:
> >
> >
> > For File and Job retentions, manual says:
> >
> > - When this time period expires, and if AutoPrune is set to yes,
> > Bacula will prune (remove) File|Job records that are older
> > than the specified File Retention period
> >
> > - The shortest retention period of the three takes precedence.
> >
> > So, I seem to recall that manual prunes do not trigger this action. I
> > could be wrong here. I have seen some complicated
> > things with retention periods and pruning in the years I have worked
> > with Bacula. :)
> >
> > Of all the people I know, my colleague Ana is the expert I refer to for
> > retention and pruning questions. :)  ANA! Where are
> > you?  :D
> >
> >
> > Also, something else to consider since I do not know your full
> > environment:
> >
> > If you have jobs on a volume from several different clients, and the
> > files/jobs of one client meet their retention period and
> > are auto-pruned, consider that other jobs from clients with longer
> > retention periods will prevent this volume from being
> > pruned/purged/recycled/re-used.
> >
> > I will ping Ana off list to see if she might chime in here.
> >
> >
> > Best regards,
> > Bill
> >
> >
> > --
> > Bill Arlofski
> > w...@protonmail.com
> >
> >
> >
> > _______________________________________________
> > Bacula-users mailing list
> > Bacula-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
> --
> ---------------------------------
> -  Nick Bright                  -
> -  KwiKom Communications        -
> -  Office 800-379-7292          -
> -  Direct 620-228-5653          -
> -  Web https://www.kwikom.com/  -
> ---------------------------------
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
Bacula-users mailing list

Reply via email to