[Bug 1770528] Re: Purging of old backup - Simple cut-off is not happening
It's a bit tricky to come up with simple description. IMHO examples are
the best way how to demonstrate the functionality, so, please find below
what I would include in the documentation/help for the Simple Backup
Suite.
~~~
Purging
Is the instrument to keep your backup lean and prevents your data storage from
overfilling in conjunction with your overall backup strategy. It's triggered at
the end of a successful backup execution. The removal of old backups can be
configured in two ways:
- Simple cut-off: Erase all backups older than "n" days.
To prevent unwanted data lost in the case when using INCREMENTAL backups,
Sbackup will only remove all the backups that are before the parent (FULL) of
the INCREMENTAL (child) backups. In other words,
the result of modulo operation of the "n" days and the frequency of FULL
backup should be 0 to achieve desired effect.
Example 1 (incorrect setting):
Do FULL backup once every 7 days ("f"=7)
Simple cut-off: Erase backups older than 10 days ("n"=10)
n mod f = 3 ... which is NOT 0.
In this case, only backups older then 14 days shall be purged (14 mod
7 = 0), because if it would done that simple cut-off, it would leave 2
INCREMENTAL (orphans) backups without their FULL (parent) backup
2018-05-01.Ubuntu.ful/ <-- CUTOFF
2018-05-02.Ubuntu.inc/ <-- CUTOFF
2018-05-03.Ubuntu.inc/ <-- CUTOFF
2018-05-04.Ubuntu.inc/ <-- CUTOFF
2018-05-05.Ubuntu.inc/ <-- CUTOFF
2018-05-06.Ubuntu.inc/
2018-05-07.Ubuntu.inc/
2018-05-08.Ubuntu.ful/
2018-05-09.Ubuntu.inc/
2018-05-10.Ubuntu.inc/
2018-05-11.Ubuntu.inc/
2018-05-12.Ubuntu.inc/
2018-05-13.Ubuntu.inc/
2018-05-14.Ubuntu.inc/
2018-05-15.Ubuntu.ful/
Example 2 (correct setup):
Do FULL backup once every 7 days ("f"=7)
Simple cut-off: Erase backups older than 7 days ("n"=7)
n mod f = 0
which means, in this case, it will keep backups up to 2 weeks. Once
the 3rd FULL backup will be completed, it shall purge the backups from the 1st
to 7th of May.
- Logarithmic
as existing...
~~~
as not sure again what it would be the behavior.
Perhaps in the case of using INCREMENTAL backups, it should keep
nearest FULL backup.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1770528
Title:
Purging of old backup - Simple cut-off is not happening
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/sbackup/+bug/1770528/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1770528] Re: Purging of old backup - Simple cut-off is not happening
Hi wattazoum, Thank you so much for looking into this issue and explaining how the Sbackup works. Well, I've slowly arrived to similar conclusion and yes, you're right about our configuration was kind of clashing. (Just inherited how it was setup and my aim is to improve it). With no description/documentation it was a bit challenge to guess what would be the real behaviour of the "Simple cut-off" option. Frustrated, I based my understanding on the explicit name of the option even though I understand it would leave us with those orphants. Also, I'd rather have it working in more smarter and safer way, which seems it doesexcept keeps disk full. You're right, my expectation would put the whole backup in the risk, OTOH I would know what to expect. I'll try to come up with some wordings for the documentation and tooltips, as the behaviour seems to be alright. So, hopefully everyone could configure Sbackup in the right way. Cheers, -d -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1770528 Title: Purging of old backup - Simple cut-off is not happening To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sbackup/+bug/1770528/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1770528] Re: Purging of old backup - Simple cut-off is not happening
Hi, I was one of the main developers of Sbackup. I have devoted myself to other projects for quite some time now (leaving sbackup to others). Developers interested in this project come and go depending on their free time. This explains the time it took for you to have and answer to your questions. I want to praise you for the effort you put on analyzing the issue. In your example, the way you expected Sbackup to work is to do the action described bellow: 2018-04-25_22.00.11.927348.mcs-Ubuntu.ful/ <-- CUTOFF 2018-04-26_22.00.08.250440.mcs-Ubuntu.inc/ <-- CUTOFF 2018-04-27_22.00.07.487993.mcs-Ubuntu.inc/ <-- CUTOFF 2018-04-28_22.00.08.383888.mcs-Ubuntu.inc/ <-- CUTOFF 2018-04-29_22.00.06.820683.mcs-Ubuntu.inc/ <-- CUTOFF 2018-04-30_22.00.08.240341.mcs-Ubuntu.inc/ 2018-05-01_22.00.08.232810.mcs-Ubuntu.inc/ 2018-05-02_22.00.07.980758.mcs-Ubuntu.ful/ 2018-05-03_22.00.07.042217.mcs-Ubuntu.inc/ 2018-05-04_22.00.08.945832.mcs-Ubuntu.inc/ 2018-05-05_22.00.07.219102.mcs-Ubuntu.inc/ 2018-05-06_22.00.08.145880.mcs-Ubuntu.inc/ 2018-05-07_22.00.08.746766.mcs-Ubuntu.inc/ 2018-05-08_22.00.08.245251.mcs-Ubuntu.inc/ 2018-05-09_22.00.07.699726.mcs-Ubuntu.ful/ However, doing this would leave 2 *orphans backups* 2018-04-30_22.00.08.240341.mcs-Ubuntu.inc/ 2018-05-01_22.00.08.232810.mcs-Ubuntu.inc/ What I call orphan backup here is a backup you would not be able to use (because you would not be able to revert a folder from it. Every Incremental backup has to have a parent (Full backup). So even with the Simple cutoff fonctionnality, we don't want you to lose the 2 incremental backups that would have been left on the disk. So Sbackup will only remove all the backups that are before the parent of the 2 inc backup. This explain the situation you got. I agree however that is not described in the documentation. I would welcome any suggestion on the way to describe it in a user friendly way. ** Changed in: sbackup (Ubuntu) Assignee: (unassigned) => Oumar Aziz OUATTARA (wattazoum) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1770528 Title: Purging of old backup - Simple cut-off is not happening To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sbackup/+bug/1770528/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1770528] Re: Purging of old backup - Simple cut-off is not happening
Interesting observation: The last night I've change the value of *Simple cut-off:* from 10 to 7, so now it looks like: Simple cut-off: Erase all backup older than 7 days. The output of the sbackup in the log is also interesting: 2018-05-16 22:30:00,509 - INFO: TAR returned a message: Total bytes written: 372029440 (355MiB, 723KiB/s) 2018-05-16 22:30:00,509 - INFO: TAR has been finished successfully. 2018-05-16 22:30:05,150 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-16 22:30:05,155 - INFO: Simple purge - remove freestanding snapshots older than 7 days. 2018-05-16 22:30:05,497 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-16 22:30:05,644 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-16 22:30:05,914 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-16 22:30:06,084 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-16 22:30:06,195 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-16 22:30:06,346 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-16 22:30:09,212 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-16 22:30:09,222 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-16 22:30:09,227 - INFO: Backup process finished. 2018-05-16 22:30:09,235 - INFO: Terminating GIO File Access Manager. 2018-05-16 22:30:09,235 - INFO: Processing of profile successfully finished (no errors) But the most interesting stuff is that it deleted some of the old backups. Before this backup was executed there were: #of ALL backups:15 #of CORRUPTED:1 #of FUL:2 #of INC:12 After this backup finished we have ended up with: #of ALL backups:9 #of CORRUPTED:1 #of FULL:1 #of INC:7 Tha last scheduled backup created another INC backup, and then the purge triggered. It seems that there is some hidden fuzzy undescribed/undocumented logic for what that purge number means. Definitely not the straightforward as it declares "erase all older than " $n "days". -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1770528 Title: Purging of old backup - Simple cut-off is not happening To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sbackup/+bug/1770528/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1770528] Re: Purging of old backup - Simple cut-off is not happening
To workaround this issue, I wrote a simple shell script located at: https://gist.github.com/delphym/62e99ef7ba936879580c66c6a9685ef3#file-clean_old_backups-sh Someone might find it helpful. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1770528 Title: Purging of old backup - Simple cut-off is not happening To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sbackup/+bug/1770528/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1770528] Re: Purging of old backup - Simple cut-off is not happening
At the end of a sbackup.log there is a record that "Simple purge" has been attempted (without any status what it actually did): 2018-05-13 16:45:25,339 - INFO: Leading '/' from member names were removed. 2018-05-13 16:45:25,350 - INFO: TAR returned a message: Total bytes written: 370298880 (354MiB, 768KiB/s) 2018-05-13 16:45:25,351 - INFO: TAR has been finished successfully. 2018-05-13 16:45:30,078 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-13 16:45:30,082 - INFO: Simple purge - remove freestanding snapshots older than 10 days. 2018-05-13 16:45:30,177 - INFO: Corrupt snapshot `2018-05-09_22.00.07.699726.mcs-Ubuntu.corrupt` found. Skipped. 2018-05-13 16:45:30,181 - INFO: Backup process finished. 2018-05-13 16:45:30,186 - INFO: Terminating GIO File Access Manager. 2018-05-13 16:45:30,186 - INFO: Processing of profile successfully finished (no errors) I was also wondering what is the definition of the "freestanding" snapshot... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1770528 Title: Purging of old backup - Simple cut-off is not happening To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sbackup/+bug/1770528/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
