Re: [Bacula-users] Retaining files indefinitely ?
Sorry for my ambiguity. I meant versions of files. I don't have access to Retrospect now to check on this point. I will re-read the documentation. At first look "backups" could be interpreted as volumes or files. My understanding was files but now I'm not so sure. Regards Chris Wilkinson On Tue, 4 Sep 2018, 2:28 p.m. Martin Simmons, wrote: > I can't decide what you mean by "the last n >=1 versions" (versions of > files > or versions of backups?). > > Yes, with m=30 you could set a volume retention time of 31 days. If you > also > set a volume use duration of 7 days, then you would have 2 volumes. > However, > I would not use n=1 because you risk losing your only backup due to media > failure or if your backup is delayed for a few days due to vacation. > > Yes, deleted files will be unrecoverable after 31 days (or probably 60 days > because the previous backup will in fact be retained until the volume is > needed for the next backup). > > I've not found any evidence that Retrospect keeps n copies of deleted files > forever. The 2 descriptions of "Staged Backup Strategy" in > https://www.retrospect.com/uk/documentation/user_guide/win/management say > that > you have a "10 day rolling window" for quick local restores and the > description of "Grooming Options for Disk Backup Sets" talks about backups > for > each source, not individual files. > > __Martin > > > > On Mon, 3 Sep 2018 21:57:38 +0100, Chris Wilkinson said: > > > > My recollection is that Retrospect retains the last n >=1 versions for > disk based backup. > > > > On the last point, to keep 1 latest copy (n=1), I could run a full job > every m=30 days and set a volume retention time of 31 days? > > > > If a file were deleted or lost for whatever reason, then wouldn’t it be > pruned out after 31 days and then be unrecoverable? > > > > The manual pages suggested seem to describe a scheme that will retain > files for up to 6 months. There wasn’t a requirement to retain for any > longer. > > > > Regards > > Chris Wilkinson > > > > > On 3 Sep 2018, at 11:29, Martin Simmons wrote: > > > > > > Does Retrospect actually retain the last version of every file > indefinitely? > > > Or does it retain the last version of every backup indefinitely? I > suspect it > > > is the latter. > > > > > > The difference is that you cannot restore a deleted file after the > last backup > > > that contained it has been deleted. > > > > > > It is easy to configure Bacula to keep the last n copies of each > Bacula job. > > > You just need to run a job with level=full every m days and set the > retention > > > periods to at least n times m. See > > > http://www.bacula.org/9.2.x-manuals/en/main/Automated_Disk_Backup.html > for > > > example. > > > > > > __Martin > > > > > > > > > > > >> On Sun, 2 Sep 2018 14:51:18 +0100, Chris Wilkinson said: > > >> > > >> Many thanks to all. > > >> > > >> I previously used Retrospect and this had a facility to set the > number of > > >> versions to be retained 1+. Earlier versions would be removed through > a > > >> scheduled "groom" job where storage space would be reclaimed. This > meant > > >> that there would always be exactly 1+ versions retained indefinitely. > > >> > > >> It doesn't appear that that behaviour can be replicated in Bacula, the > > >> nearest being to retain everything always. > > >> > > >> Regards > > >> Chris Wilkinson > > >> > > >> On Wed, 29 Aug 2018, 1:52 p.m. Dan Langille, > wrote: > > >> > > On Aug 28, 2018, at 2:54 AM, Chris Wilkinson < > winstonia...@gmail.com> > > >>> wrote: > > > > One of my requirements for backups is that the last version of every > > >>> file should remain in the backup storage indefinitely or at least > for a > > >>> long time. > > > > Could I achieve this by writing "File Retention = 10 years" in the > pool > > >>> or client definition? > > >>> > > >>> Yes. But you'll also need Volume and Job retention also set to 10 > years if > > >>> you want to easily restore that data. > > >>> > > >>> This will keep all records, not just the latest. > > >>> > > >>> Yes, this will use more database records, but my view is that this > is a > > >>> cheap price to pay. > > >>> > > >>> Also, as mentioned by others, does this requirement mean that a file > > >>> deleted 6 years ago needs to be kept for another 4 years? > > >>> > > >>> You'll need to clarify that 10 years meets the "indefinitely" > requirement. > > >>> > > >>> Given the originally stated requirements, it seems your only > solution is > > >>> not to ever recycle volumes or allow the Catalog to be pruned. > > >>> > > My concern is that, if this is possible, then it would stop > automatic > > >>> volume truncation and recycling and eventually result in out of space > > >>> storage errors. > > >>> > > >>> Yes, you need to have enough storage. > > >>> > > >>> > > >>> -- > > >>> Dan Langille - BSDCan / PGCon > > >>> d...@langille.org > > >>> > > >>> > > >>> > > >> > > > > > --
Re: [Bacula-users] Retaining files indefinitely ?
I can't decide what you mean by "the last n >=1 versions" (versions of files or versions of backups?). Yes, with m=30 you could set a volume retention time of 31 days. If you also set a volume use duration of 7 days, then you would have 2 volumes. However, I would not use n=1 because you risk losing your only backup due to media failure or if your backup is delayed for a few days due to vacation. Yes, deleted files will be unrecoverable after 31 days (or probably 60 days because the previous backup will in fact be retained until the volume is needed for the next backup). I've not found any evidence that Retrospect keeps n copies of deleted files forever. The 2 descriptions of "Staged Backup Strategy" in https://www.retrospect.com/uk/documentation/user_guide/win/management say that you have a "10 day rolling window" for quick local restores and the description of "Grooming Options for Disk Backup Sets" talks about backups for each source, not individual files. __Martin > On Mon, 3 Sep 2018 21:57:38 +0100, Chris Wilkinson said: > > My recollection is that Retrospect retains the last n >=1 versions for disk > based backup. > > On the last point, to keep 1 latest copy (n=1), I could run a full job every > m=30 days and set a volume retention time of 31 days? > > If a file were deleted or lost for whatever reason, then wouldn’t it be > pruned out after 31 days and then be unrecoverable? > > The manual pages suggested seem to describe a scheme that will retain files > for up to 6 months. There wasn’t a requirement to retain for any longer. > > Regards > Chris Wilkinson > > > On 3 Sep 2018, at 11:29, Martin Simmons wrote: > > > > Does Retrospect actually retain the last version of every file indefinitely? > > Or does it retain the last version of every backup indefinitely? I suspect > > it > > is the latter. > > > > The difference is that you cannot restore a deleted file after the last > > backup > > that contained it has been deleted. > > > > It is easy to configure Bacula to keep the last n copies of each Bacula job. > > You just need to run a job with level=full every m days and set the > > retention > > periods to at least n times m. See > > http://www.bacula.org/9.2.x-manuals/en/main/Automated_Disk_Backup.html for > > example. > > > > __Martin > > > > > > > >> On Sun, 2 Sep 2018 14:51:18 +0100, Chris Wilkinson said: > >> > >> Many thanks to all. > >> > >> I previously used Retrospect and this had a facility to set the number of > >> versions to be retained 1+. Earlier versions would be removed through a > >> scheduled "groom" job where storage space would be reclaimed. This meant > >> that there would always be exactly 1+ versions retained indefinitely. > >> > >> It doesn't appear that that behaviour can be replicated in Bacula, the > >> nearest being to retain everything always. > >> > >> Regards > >> Chris Wilkinson > >> > >> On Wed, 29 Aug 2018, 1:52 p.m. Dan Langille, wrote: > >> > On Aug 28, 2018, at 2:54 AM, Chris Wilkinson > >>> wrote: > > One of my requirements for backups is that the last version of every > >>> file should remain in the backup storage indefinitely or at least for a > >>> long time. > > Could I achieve this by writing "File Retention = 10 years" in the pool > >>> or client definition? > >>> > >>> Yes. But you'll also need Volume and Job retention also set to 10 years if > >>> you want to easily restore that data. > >>> > >>> This will keep all records, not just the latest. > >>> > >>> Yes, this will use more database records, but my view is that this is a > >>> cheap price to pay. > >>> > >>> Also, as mentioned by others, does this requirement mean that a file > >>> deleted 6 years ago needs to be kept for another 4 years? > >>> > >>> You'll need to clarify that 10 years meets the "indefinitely" requirement. > >>> > >>> Given the originally stated requirements, it seems your only solution is > >>> not to ever recycle volumes or allow the Catalog to be pruned. > >>> > My concern is that, if this is possible, then it would stop automatic > >>> volume truncation and recycling and eventually result in out of space > >>> storage errors. > >>> > >>> Yes, you need to have enough storage. > >>> > >>> > >>> -- > >>> Dan Langille - BSDCan / PGCon > >>> d...@langille.org > >>> > >>> > >>> > >> > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Retaining files indefinitely ?
My recollection is that Retrospect retains the last n >=1 versions for disk based backup. On the last point, to keep 1 latest copy (n=1), I could run a full job every m=30 days and set a volume retention time of 31 days? If a file were deleted or lost for whatever reason, then wouldn’t it be pruned out after 31 days and then be unrecoverable? The manual pages suggested seem to describe a scheme that will retain files for up to 6 months. There wasn’t a requirement to retain for any longer. Regards Chris Wilkinson > On 3 Sep 2018, at 11:29, Martin Simmons wrote: > > Does Retrospect actually retain the last version of every file indefinitely? > Or does it retain the last version of every backup indefinitely? I suspect it > is the latter. > > The difference is that you cannot restore a deleted file after the last backup > that contained it has been deleted. > > It is easy to configure Bacula to keep the last n copies of each Bacula job. > You just need to run a job with level=full every m days and set the retention > periods to at least n times m. See > http://www.bacula.org/9.2.x-manuals/en/main/Automated_Disk_Backup.html for > example. > > __Martin > > > >> On Sun, 2 Sep 2018 14:51:18 +0100, Chris Wilkinson said: >> >> Many thanks to all. >> >> I previously used Retrospect and this had a facility to set the number of >> versions to be retained 1+. Earlier versions would be removed through a >> scheduled "groom" job where storage space would be reclaimed. This meant >> that there would always be exactly 1+ versions retained indefinitely. >> >> It doesn't appear that that behaviour can be replicated in Bacula, the >> nearest being to retain everything always. >> >> Regards >> Chris Wilkinson >> >> On Wed, 29 Aug 2018, 1:52 p.m. Dan Langille, wrote: >> On Aug 28, 2018, at 2:54 AM, Chris Wilkinson >>> wrote: One of my requirements for backups is that the last version of every >>> file should remain in the backup storage indefinitely or at least for a >>> long time. Could I achieve this by writing "File Retention = 10 years" in the pool >>> or client definition? >>> >>> Yes. But you'll also need Volume and Job retention also set to 10 years if >>> you want to easily restore that data. >>> >>> This will keep all records, not just the latest. >>> >>> Yes, this will use more database records, but my view is that this is a >>> cheap price to pay. >>> >>> Also, as mentioned by others, does this requirement mean that a file >>> deleted 6 years ago needs to be kept for another 4 years? >>> >>> You'll need to clarify that 10 years meets the "indefinitely" requirement. >>> >>> Given the originally stated requirements, it seems your only solution is >>> not to ever recycle volumes or allow the Catalog to be pruned. >>> My concern is that, if this is possible, then it would stop automatic >>> volume truncation and recycling and eventually result in out of space >>> storage errors. >>> >>> Yes, you need to have enough storage. >>> >>> >>> -- >>> Dan Langille - BSDCan / PGCon >>> d...@langille.org >>> >>> >>> >> -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Retaining files indefinitely ?
Does Retrospect actually retain the last version of every file indefinitely? Or does it retain the last version of every backup indefinitely? I suspect it is the latter. The difference is that you cannot restore a deleted file after the last backup that contained it has been deleted. It is easy to configure Bacula to keep the last n copies of each Bacula job. You just need to run a job with level=full every m days and set the retention periods to at least n times m. See http://www.bacula.org/9.2.x-manuals/en/main/Automated_Disk_Backup.html for example. __Martin > On Sun, 2 Sep 2018 14:51:18 +0100, Chris Wilkinson said: > > Many thanks to all. > > I previously used Retrospect and this had a facility to set the number of > versions to be retained 1+. Earlier versions would be removed through a > scheduled "groom" job where storage space would be reclaimed. This meant > that there would always be exactly 1+ versions retained indefinitely. > > It doesn't appear that that behaviour can be replicated in Bacula, the > nearest being to retain everything always. > > Regards > Chris Wilkinson > > On Wed, 29 Aug 2018, 1:52 p.m. Dan Langille, wrote: > > > > On Aug 28, 2018, at 2:54 AM, Chris Wilkinson > > wrote: > > > > > > One of my requirements for backups is that the last version of every > > file should remain in the backup storage indefinitely or at least for a > > long time. > > > > > > Could I achieve this by writing "File Retention = 10 years" in the pool > > or client definition? > > > > Yes. But you'll also need Volume and Job retention also set to 10 years if > > you want to easily restore that data. > > > > This will keep all records, not just the latest. > > > > Yes, this will use more database records, but my view is that this is a > > cheap price to pay. > > > > Also, as mentioned by others, does this requirement mean that a file > > deleted 6 years ago needs to be kept for another 4 years? > > > > You'll need to clarify that 10 years meets the "indefinitely" requirement. > > > > Given the originally stated requirements, it seems your only solution is > > not to ever recycle volumes or allow the Catalog to be pruned. > > > > > My concern is that, if this is possible, then it would stop automatic > > volume truncation and recycling and eventually result in out of space > > storage errors. > > > > Yes, you need to have enough storage. > > > > > > -- > > Dan Langille - BSDCan / PGCon > > d...@langille.org > > > > > > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Retaining files indefinitely ?
Many thanks to all. I previously used Retrospect and this had a facility to set the number of versions to be retained 1+. Earlier versions would be removed through a scheduled "groom" job where storage space would be reclaimed. This meant that there would always be exactly 1+ versions retained indefinitely. It doesn't appear that that behaviour can be replicated in Bacula, the nearest being to retain everything always. Regards Chris Wilkinson On Wed, 29 Aug 2018, 1:52 p.m. Dan Langille, wrote: > > On Aug 28, 2018, at 2:54 AM, Chris Wilkinson > wrote: > > > > One of my requirements for backups is that the last version of every > file should remain in the backup storage indefinitely or at least for a > long time. > > > > Could I achieve this by writing "File Retention = 10 years" in the pool > or client definition? > > Yes. But you'll also need Volume and Job retention also set to 10 years if > you want to easily restore that data. > > This will keep all records, not just the latest. > > Yes, this will use more database records, but my view is that this is a > cheap price to pay. > > Also, as mentioned by others, does this requirement mean that a file > deleted 6 years ago needs to be kept for another 4 years? > > You'll need to clarify that 10 years meets the "indefinitely" requirement. > > Given the originally stated requirements, it seems your only solution is > not to ever recycle volumes or allow the Catalog to be pruned. > > > My concern is that, if this is possible, then it would stop automatic > volume truncation and recycling and eventually result in out of space > storage errors. > > Yes, you need to have enough storage. > > > -- > Dan Langille - BSDCan / PGCon > d...@langille.org > > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Retaining files indefinitely ?
> On Aug 28, 2018, at 2:54 AM, Chris Wilkinson wrote: > > One of my requirements for backups is that the last version of every file > should remain in the backup storage indefinitely or at least for a long time. > > Could I achieve this by writing "File Retention = 10 years" in the pool or > client definition? Yes. But you'll also need Volume and Job retention also set to 10 years if you want to easily restore that data. This will keep all records, not just the latest. Yes, this will use more database records, but my view is that this is a cheap price to pay. Also, as mentioned by others, does this requirement mean that a file deleted 6 years ago needs to be kept for another 4 years? You'll need to clarify that 10 years meets the "indefinitely" requirement. Given the originally stated requirements, it seems your only solution is not to ever recycle volumes or allow the Catalog to be pruned. > My concern is that, if this is possible, then it would stop automatic volume > truncation and recycling and eventually result in out of space storage errors. Yes, you need to have enough storage. -- Dan Langille - BSDCan / PGCon d...@langille.org -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Retaining files indefinitely ?
Do you mean, after the file has been deleted from the system? IMHO that's a job for an archiving solution, not for a backup system. On Tue, Aug 28, 2018, at 08:54, Chris Wilkinson wrote: > One of my requirements for backups is that the last version of every > file should remain in the backup storage indefinitely or at least for > a long time.> > Could I achieve this by writing "File Retention = 10 years" in the > pool or client definition?> > My concern is that, if this is possible, then it would stop automatic > volume truncation and recycling and eventually result in out of space > storage errors.> > Regards > Chris Wilkinson > -- Tilman Schmidt til...@imap.cc -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Retaining files indefinitely ?
Hello, wt., 28 sie 2018 o 08:56 Chris Wilkinson napisał(a): > One of my requirements for backups is that the last version of every file > I'm not a native English speaker, so I need an explanation here. What is your understanding of the "last version". I understand it as the recent one (the opposite to the first one), so every time you change the file it become "the last version". > should remain in the backup storage indefinitely or at least for a long > time. > > It sounds to me as you want to archive a single file version in your long time retention storage. > Could I achieve this by writing "File Retention = 10 years" in the pool or > client definition? > > File Retention parameter applies to catalog records only. The Volume retention is a parameter which preserve backup data from recycling. > My concern is that, if this is possible, then it would stop automatic > volume truncation and recycling and eventually result in out of space > storage errors. > No, the Volume Retention parameter has precedence to File Retention and Job Retention, so you can set any insanely long time value to FR/JR parameters (i.e. 100Y) and your backup will be archived no longer then VR parameter. best regards -- Radosław Korzeniewski rados...@korzeniewski.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Retaining files indefinitely ?
One of my requirements for backups is that the last version of every file should remain in the backup storage indefinitely or at least for a long time. Could I achieve this by writing "File Retention = 10 years" in the pool or client definition? My concern is that, if this is possible, then it would stop automatic volume truncation and recycling and eventually result in out of space storage errors. Regards Chris Wilkinson -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
