bob, thank you for you effort to help! Response part (4) was relevant for me. I solved the mystery by now: VOlume sin state Error do not have an expiry date and so the associated jobs also do not get pruned.
> On 21. Oct 2024, at 16:08, Rob Gerber <r...@craeon.net> wrote: > > Justin, > > I don't have answers to all your questions (really, to any of your specific > questions, anyhow). I do have some thoughts, though. > > Please keep in mind that compared to many on this list, I'm a bacula novice. > I might get something wrong. I also haven't entirely understood the problems > you're facing, so maybe some of this might sound irrelevant. Please ignore if > so. > > However, I think there might be a few things I can help with here. > > 1. I know that bacula can define 3 kinds of retention: File, Job, and Volume. > File retention means "how long should I keep records about these files in the > catalog", NOT "how long should I keep these files in the bacula backup > volumes". So if you prune file retention records you will delete information > about which files are in a job, but will still be able to restore that job > later. Maybe you want to save catalog space for older backup jobs. > The consequences of pruning job and volume records are different. A > previously used volume with no jobs in the catalog is subject to recycling > and reuse. > > 2. There are default retention periods coded into bacula that will be used if > the administrator does not specify their own retention periods. I don't know > what they are, but you can look them up. I think maybe 60 days? Not sure. > > 3. To my knowledge, bacula stores File, Job, and Volume retention time > periods in the director configuration file (bacula-dir.conf). The manual can > be more specific about this, but at minimum you should be able to specify > these options under either the jobdef, job, or pool resource (I think pool > resource maybe can't use file retention, only job or volume retention?). > Maybe you can also specify retention periods at the FD level, but I haven't > used that and am not sure if it's possible or how it works. I would think > that the FD probably wouldn't be able to specify, say, volume retention time > since I imagine that's outside the scope of the FD. > ****Retention periods specified in the pool over-ride all other retention > periods!**** > > 4. If bacula doesn't store any file, job, or volume retention times in the > catalog, it's probably to save on database space. From a programming > perspective, there's no reason to store retention times in the catalog. > Bacula can simply look at the relevant configuration files, check the current > time/date, and then check the catalog for file, job, or volumes that are > older than the specified retention period for those items. This wouldn't save > a lot of database space for jobs or volumes, but file retention... oh boy. > Yeah, for file retention it'd save a LOT of database space. Think of all the > posix compliant time/date codes, one for every copy of a file ever backed up. > Those bytes add up over millions and billions of file entries. > > Regards, > Robert Gerber > 402-237-8692 > r...@craeon.net <mailto:r...@craeon.net> > > On Sun, Oct 20, 2024 at 6:15 AM Justin Case <jus7inc...@gmail.com > <mailto:jus7inc...@gmail.com>> wrote: > I have browsed the catalog database a bit more and I saw one thing that might > explain the problem I am having, but I don't have sufficient technical > background on the Bacula implementation details of different versions. > > I am running bacula-dir 15.0.2 and bacularis 3.1.0. The Bacula manual and > Bacularis offer “Job Retention” and “File Retention” in the pool resource, > supposedly overriding the same fields in the client resource. I am not using > these fields in the client resource, and in the client table there are the > job and file retention default values. > I am only using the job and file retention fields in the pool resources in > Bacularis. > What irritates me is that in the pool DB table there are no columns for job > and for file retention! > Are these values stored elsewhere or has my DB not been initially set up or > not correctly upgraded? > > If the pool resource retention values for jobs and files are missing in my DB > this would mean that bacula-dir is using the values from the client > resources, which is consistent with what I am seeing (with 2 exceptions): > The oldest jobs are 6 months old (job retention default value). Exceptions: > (a) catalog backup jobs go back to the date when the installation was set up > (b) two other jobs have altogether 4 instances that are older than 6 months. > > So open questions for me are: > > (1) where are job retention and file retention stored for the pool resource? > > (2) if in the pool table, why does my pool table not have corresponding > columns for job and file retention? How may I fix this? > > (3) why are there catalog backup jobs older than the default job retention of > 180d in my catalog, they should have been expired? > > (4) why are there 4 other job instances older than 180d? > > Bonus questions: > > (5) it seems that only jobs of types Backup, Copy Job and Copy (Migration > also? I dont use those) are pruned. Is this the correct behavior? > > (6) Is there a builtin mechanism to expire and prune Admin jobs and Verify > jobs? > > > >> On 20. Oct 2024, at 12:21, Justin Case <jus7inc...@gmail.com >> <mailto:jus7inc...@gmail.com>> wrote: >> >> Hi Bill, >> >> please find my response inline below: >> >>> On 19. Oct 2024, at 22:59, Bill Arlofski <w...@protonmail.com >>> <mailto:w...@protonmail.com>> wrote: >>> >>> On Wednesday, October 16th, 2024 at 06:15, Justin Case >>> <jus7inc...@gmail.com <mailto:jus7inc...@gmail.com>> wrote: >>> >>>> I am wondering why I am seeing in my catalog lots of jobs older than the >>>> JobRetention defined in the pools, and also older than the default >>>> JobRetention assumed for the clients. >>>> The volume recycling seems to work fine adhering to the VolumeRetention in >>>> the pools. >>>> >>> >>>> To me it is a mystery, probably be cause I overlook some dependencies I am >>>> not aware of. >>>> Can someone please help me understanding this. >>> >>> Hello Justin, >>> >>> From the rest of this thread, it appears that your prunining is working as >>> configured/expected. >> >> I doubt that, at least no as intended. >> >>> Where are you seeing the old jobs? bconsole? BWeb? Bacularis? >> >> Bacularis. >> >>> They could be coming from the `jobhisto` table. >>> >>> And you can prove this by querying the `job` and `jobhisto` tables and >>> compare the results. >> >> job has all the jobs >> jobhisto is empty >> >> Still sounds to me like job expiry not work as intended - maybe config is >> wrong, but I don’t understand where the mistake is. You find the config >> excerpt in my initial post. >> >> Can you help? >> >> All the best, >> J/C > > _______________________________________________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net <mailto:Bacula-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/bacula-users > <https://lists.sourceforge.net/lists/listinfo/bacula-users>
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
