Hi Rob, I understood well since our first exchange that the MaximumVolumes at 1 was a heresy. I also understood well that to best manage several volumes, it remained important to separate Full, Diff and Inc in different pools. So, we are returning substantially to what I already have. On the other hand, sincerely, I do not see what it would bring me in terms of occupied space to group the Full of my FDs in a single pool instead of having a pool per FD, the same for Diff and Inc.
What I know is that 2 Full Volumes + 3 (or 4) Diff Volumes + 2 Volume of 6 days of Inc, for the 2 FDs that I save on this SD (the 4 other FDs are saved on another SD), it will be very tight with the space that I have. And I cannot unfortunately increase the space because it is a hosted server whose increase in space is too expensive. I would have preferred to avoid getting into something more complicated by forcing the expiration of volumes by admin job or something else. I need to find a concrete solution to save my 30 days of 2 FDs on this SD. I must also admit that I don't know what to put optimally in: File Retention = ?? days Job Retention = ?? days for clients. In any case, thank you for all the efforts you make to help me. Even if I'm not sure I understand everything. I'm still thinking about all this and if anyone else wants to shed some light, don't hesitate. Regarding ZSTD, I confirm that it is already installed, called by many packages. I configured bacula to use it. Le mercredi 08 janvier 2025 à 17:27 -0600, Rob Gerber a écrit : > Replies below. :) > Regards, > Robert Gerber > 402-237-8692 > r...@craeon.net > > > On Wed, Jan 8, 2025 at 4:03 PM Christophe PEREZ via Bacula-users > <bacula-users@lists.sourceforge.net> wrote: > > Le mercredi 08 janvier 2025 à 14:46 -0600, Rob Gerber a écrit : > > > Christophe, > > > > Hi Rob, > > > > > > Not 2nd AND 5th, but 2nd TO 5th. > > > > That makes more sense. I understand now. > > > > > > > MaximumVolumes="2" can only work if you have only 1 FD. Really, > > > for > > > this to work the right way you must have MaximumVolumes=X where X > > > = > > > (total FD x 2). > > > > And save all FD in the same pool ? > > > > I would want to put Full backups in the Full pool, Diff in the diff > pool, etc. Different pool for different backup type and different > retention, but not different pools for different FD unless you have a > very good reason to do it. So, per storage medium, you will have 3 > total pools, Full, Diff, Inc. You do not want to mix media types in a > pool, so if you were backing up to cloud, to another SD, or to tape > you would want a Full, Diff, and Inc pool for each storage medium or > device. In my 15.0.2 device, I have local backup pools Synology- > Local-Full, Synology-Local-Diff, and Synology-Local-Inc, and pools > B2-Full, B2-Diff, B2-Inc for copying jobs from the local pools to the > Backblaze B2 cloud service. > > It seems to me that this will not gain me any space. > > > > > > I am trying to say that for you to use the MaximumVolumes=X idea > with MaximumVolumeJobs=1, X must be >= the number of volumes you > expect to use within your retention period. Let us say you have 6 FD, > MaximumVolumes=6, MaximumVolumeJobs=1, VolumeRetention=40days, and > you do 1 full backup each month. In month 1 there is no problem - > bacula does its backups and creates 6 volumes. In month 2, Bacula > cannot make any volumes available without deleting the volumes from > last month BEFORE you have done any new backups! In my understanding > bacula would not violate the retention period, so it would be unable > to make the new backups. So in this example you would want at least > MaximumVolumes=12 with MaximumVolumeJobs=1. You don't want to run out > of volumes before your retention period expires. > > There are other ways to limit bacula's space usage. I'll talk about > them. > > > > So, with Maximum Volume Jobs = 0, the volume size will continue to > > grow? > > > > I don't know what would happen if we set MaximumVolumeJobs=0 , but if > we comment it out or remove it, yes, the volume will grow forever > unless restrained in some other way. > Bacula gives us many ways to restrain volume growth. We can set > MaximumVolumeUseDuration, MaximumVolumeJobs, MaximumVolumeBytes, > etc. > See the manual for some more discussion of each. > https://www.bacula.org/15.0.x-manuals/en/main/Configuring_Director.html#SECTION00231600000000000000000 > > So I have to find something else... > > I thought I had made my life easier like that. > > > > I think MaximumVolumeJobs is good for your situation. Maybe > MaximumVolumes is difficult if not set correctly. If you experience > unexpected blockages where a job cannot run because the pool does not > have any more volumes, then you can experiment with removing > MaximumVolumes and instead relying on the VolumeRetention period to > remove volumes. > > As a warning if relying on VolumeRetention alone, it is possible that > bacula will not remove a volume if it does not have to, even if > expired. Bacula's goal is to keep your volumes available as long as > possible, just in case you need them. If that becomes a problem, I > believe you can Schedule a job of Type=Admin and tell it to truncate > expired volumes. I have not set up such a job, so I am not certain > how it is done. > > I think MaximumVolumes could work for you, as long as it is set to be > >= the number of volumes you will need during your retention period. > > Here is the bacula manual entry all about automatic volume recycling. > It is very useful. > https://www.bacula.org/15.0.x-manuals/en/main/Automatic_Volume_Recycling.html > > One extract from that link above that explains what bacula will do if > not restrained in some way: > " A key point mentioned above, that can be a source of frustration, > is that Bacula will only recycle purged Volumes if there is no other > appendable Volume available, otherwise, it will always write to an > appendable Volume before recycling even if there are Volumes marked > as Purged. This preserves your data as long as possible. So, if you > wish to “force” Bacula to use a purged Volume, you must first ensure > that no other Volume in the Pool is marked Append. If necessary, you > can manually set a volume to Full. The reason for this is that Bacula > wants to preserve the data on your old Volumes (even though purged > from the catalog) as long as absolutely possible before overwriting > it. There are also a number of directives such as Volume Use Duration > that will automatically mark a volume as Used and thus no longer > appendable." > > This behavior is why we must restrain bacula. Your suggestion of > MaximumVolumeJobs=1 and > MaximumVolumes=$expectedNumberOfVolumesNeededDuringRetentionPeriod > accomplishes this, with the warning that you will not be able to make > new backups if you run out of volumes because you added more FDs but > did not increase MaximumVolumes, or something like that. > > I have set a very short retention period on my test system, with a > job scheduled once an hour. I'll examine the behavior I see and > report back. > > > I have bacula 15.0.2. > > What linux tool does this ZSTD compression format depend on? > > > > I don't know what tool it uses. I am also running bacula 15.0.2, on > Rocky Linux 9. For me, the ZSTD feature just worked. I think bacula > made sure the correct libraries are present. > > -- Christophe PEREZ _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
